ISBN:9789899729209 Title:EuroITV2011AdjunctProceedings Editors:Damsio,ManuelJos;Cardoso,Gustavo;Quico,Clia;Geerts,David Date:20110401 Publisher:COFAC/UniversidadeLusfonadeHumanidadeseTecnologias

2

PREFACEDearEuroITV2011participants, It is with great pleasure that, on behalf of Universidade Lusfona de Humanidades e Tecnologias/ CICANT Center for Research in Applied Communications, Culture and NewTechnologiesandLINILisbonInternetandNetworksInstitute,wewelcomeyou toourinstitutionsandthecityofLisbon.Ourinstitutionsassociatedthemselvestothis eventconvincedoftherelevanceofitforreflectionanddevelopmentofthestateof theartarounddigitaltelevisionandassociatedapplications.Theevolutionoftelevision andmediahasbeenacentraltopicofresearchandtrainingatourinstitutions,anditis our belief that an organization like the EuroITV, with the high scientific quality that characterizesit,isanundeniablecontributiontotheadvancementofknowledgeinthis fieldandtheshapingofacommunitariandynamicaroundthistopic. EuroITVisincreasinglycompetitiveintermsofpapersubmissionandacceptancerate. Thisyear,wereceivedapproximately200papersubmissionsrelevantforthemainand adjunct proceedings of this conference from 24 countries. Through peer reviewing processbyprogramcommitteeandexperts,weacceptedfortheadjunctproceedings 6demos,8doctoralconsortiumproposals,8posters,12industrialcasestudiespapers and24workshoppapers.Webelievetohavecreatedarichincontentandhighquality technicalprogramspanningthreefulldays,coveringfourdifferenttracks,representing theactivitiesinTVresearcharea.PresentationsincludekeynotespeechesbyJonathan Taplin from University of Southern California "Long Time Coming; Has Interactive TV Finally Arrived?, Fernando Pereira from Instituto Superior Tcnico "Visual Compression:theFoundationalTechnologyforBetterTVExperiences"andAlKovalick fromAVID"TheMediaCloudanditsFuture".

We would like to thank the authors for providing the content of the program and all members of the organizing committee for their dedication to the success of EuroITV2011 and timely review of the submissions. We would like to acknowledge AVID, Caixa Geral de Depsitos, Fundao Gulbenkian, Fundao para a Cincia e 3

Tecnologia, Instituto de Cinema e Audiovisual, Gabinete para os Meios de ComunicaoSocial,FundaoLusoAmericana,AbreuandACMfortheirsponsorships andsupport.Wealsothankourmediapartners,RTPandtheInternationalJournalof Digital Television. Finally, we are grateful to ourcolleagues at Universidade Lusfona and LINI who worked in the organization of this event. It is therefore with great pleasure that we collaborate with EuroITV and wish you a fruitful event that is not depletedinthesedaysoftheconferencebutthatopensupspacesandopportunities fornewcollaborationsandscientificdialogues. GustavoCardoso EUROITV11GeneralChair LINI,PT DavidGeerts EUROITV11ProgramChair K.U.Leuven,BE

ManuelJosDamsio EUROITV11GeneralChair UniversidadeLusfona,PT CliaQuico EUROITV11ProgramChair UniversidadeLusfona,PT

4

TABLEOFCONTENTSPREFACE .............................................................................................................................................. 3 TABLEOFCONTENTS ........................................................................................................................... 5 Keynotes ............................................................................................................................................. 8 Demos ................................................................................................................................................12 NScreenLiveBaseballGameWatchingSystem:NovelInteractionConceptswithinaPublicSetting..13 Extractionof ContextualWeb Informationfrom TVVideo ..........................................................15 AutomaticMeasurementofPlayoutDifferencesforSocialTV,InteractiveTV,GamingandInter destinationSynchronization ............................................................................................................18 UserInterfaceToolkitforUbiquitousTV..........................................................................................20 Demo:usingspeechrecognitionforinsituvideotagging .................................................................22 Valueaddedservicesandidentificationsystem:anapproachtoelderlyviewers ............................... 24 DoctoralConsortium ..........................................................................................................................26 Researchfor Developmentof Value AddedServicesforConnectedTV...................................27 Collaborationin BroadcastMedia and Content .........................................................................31 TelevisualLeisureExperiencesofDifferentGenerationsofBasqueSpeakers ....................................35 MobileTV:TowardsaTheoryforMobileTelevision .........................................................................39 EnhancingandEvaluatingtheUserExperienceofInteractiveTVSystemsandtheirInteraction Techniques .....................................................................................................................................43 SubjectiveQualityAssessmentofFreeViewpointVideoObjects ......................................................47 AllocationAlgorithmsforInteractiveTVAdvertisements .................................................................51 Videoaccessandinteractionbasedonemotions ............................................................................55 Posters...............................................................................................................................................59 OnlineiTVusebyolderpeople:preliminaryfindingsofarapidethnographicalstudy........................60 MultipleyeConcurrentInformationDeliveryon PublicDisplays ............................................64 OlderAdultsandDigitalInteractiveTelevision:UseofaWiiController ............................................68 PredictingWhere,WhenandWhatPeopleWillWatchonTVBasedontheirPastViewingHistory ....72 UnusualCoProduction:OnlineCoCreationinCrossMediaFormatDevelopment ............................76 TrendyEpisodeDetectionataVeryShortTimeGranularityforIntelligentVODService:ACaseStudy ofLiveBaseballGame .....................................................................................................................80

5

SpatialTilingAndStreaminginanImmersiveMediaDeliveryNetwork .............................................84 Inclusionof multipledevices, languages andadvertisementsin iFanzy, a personalizedEPG .......................................................................................................................................................88 ITVinIndustry ....................................................................................................................................92 ConvergenceofTelevisedContentandGame ..................................................................................93 heckle.atTV ....................................................................................................................................94 Lettheaudiencedirect ....................................................................................................................95 Rend ezVous .....................................................................................................................................96 Settop,overthetop,future!...........................................................................................................97 SmartTVandhowtodoitright.......................................................................................................99 InnovatingUsability ......................................................................................................................100 Theubiquitousremotecontrol. .....................................................................................................101 ContentplusInteractivityas aKeyDifferentiator...........................................................................102 ITVstrategy:TheuseofDirectandIndirectCommunicationasastrategyforthecreationof interactivescripts..........................................................................................................................103 AmbientMediaEcosystemsforTVAforecast2013......................................................................105 Tutorials ...........................................................................................................................................106 DesigningandEvaluatingSocialVideoandTelevision.....................................................................107 HowtoinvestigatetheQualityofUserExperienceforUbiquitousTV? ..........................................109 DeployingSocialTV:Content,Connectivity,andCommunication ...................................................111 Workshops .......................................................................................................................................113 Workshop1:QualityofExperienceforMultimediaContentSharing:UbiquitousQoEAssessment andSupport................................................................................................................................114 QualityofExperienceofMultimediaServices:Past,Present,andFuture...................................... 115 InternetTVArchitecture.................................................................................................................. 120 BasedonScalableVideoCoding ...................................................................................................... 120 Adaptivetestingforvideoqualityassessment ............................................................................... 128 Aligningsubjectivetestsusingalowcostscommonset................................................................. 132 ImpactofReducedQualityEncodingonObjectIdentificationinStereoscopicVideo ................... 136 Impact of DisturbanceLocationson Video QualityofExperience ..................................... 140 Workshop2:FutureTV2011:MakingTelevisionPersonal&Social ............................................144

6

AnalysisoftheInformationValueofUserConnectionsforVideoRecommendationsinaSocial Network .......................................................................................................................................... 145 EmployingUserAssignedTagstoProvidePersonalizedaswellasCollaborativeTV Recommendations .......................................................................................................................... 145 SocialandInteractiveTV:AnOutsideInApproach .................................................... 145 ........................... 145 145

AnalyzingTwitterforSocialTV:SentimentExtractionforSports

OurTV:CreatingMediaContentsCommunitiesthroughRealWorldInteractions.....

ITVservicesforsocializinginpublicplaces ..................................................................................... 145 Ubeel:GeneratingLocalNarrativesforPublicDisplaysfromTaggedandAnnotatedVideoContent ........................................................................................................................................................ 145 HybridalgorithmsforrecommendingnewitemsinpersonalTV ............................ 145

MiningKnowledgeTV:AProposalforDataIntegrationintheKnowledgeTVEnvironment ......... 145 Workshop3:InteractiveDigitalTVinEmergentEconomies.......................................................146 GEmPTV:GingaNCLEmulatorforPortableDigitalTV ................................................................... 147 BusinessProcessModelinginUMLforInteractiveDigitalTelevision............................................. 151 Guidelinesforthecontentproductionoftlearning ...................................................................... 155 EvaluationofaninteractiveTVservicetoreinforcedentalhygieneeducationinchildren ........... 159 ExperiencesinDesigningandImplementinganExtensionAPItoConvergeiDTVandHomeGateway ........................................................................................................................................................ 163 AnApproachforContentPersonalizationofContextSensitiveInteractiveTVApplications ......... 169 AFrameworkArchitectureforDigitalGamestotheBrazilianDigitalTelevision............................ 171 EuroITV2011OrganizingCommittee ..........................................................................................176

7

Keynotes

8

JonathanTaplin (AnnenbergInnovationLabUniversityofSouthernCalifrnia) Jonathan Taplin is a Professor at the Annenberg School for Communication at the University of Southern California. Taplin is the Managing Director of the USC Annenberg Innovation Lab (http://www.annenberglab.com/) and also blogs at

http://jontaplin.com/,aboutwhichCoryDoctorowofBoing,Boing said,"Taplin'sblogisaseclecticasheis,astraightupanalysisblog thatripsintotheheadlines,illuminatingeverythingfromeconomic newstothewriters'striketoheavyweathertodemocraticpolitics. Taplin'sareasofspecializationareininternationalcommunicationmanagementandthefield of digital media entertainment. Taplin began his entertainment career in 1969 as Tour ManagerforBobDylanandTheBand.In1973heproducedMartinScorsese'sfirstfeaturefilm, MeanStreetswhichwasselectedfortheCannesFilmFestival.Between1974and1996,Taplin produced 26 hours of television documentaries (including The Prize and Cadillac Desert for PBS)and12featurefilmsincludingTheLastWaltz,UntilTheEndoftheWorld,UnderFireand ToDieFor.HisfilmswerenominatedforOscarandGoldenGlobeawardsandchosenforThe CannesFilmFestivalseventimes. In1984TaplinactedastheinvestmentadvisortotheBassBrothersintheirsuccessfulattempt to save Walt Disney Studios from a corporate raid. This experience brought him to Merrill Lynch, where he served as vice president of media mergers and acquisitions. In this role, he helped reengineer the media landscape on transactions such as the leveraged buyout of Viacom.TaplinwasafounderofIntertainerandhasservedasitsChairmanandCEOsinceJune 1996. Intertainer was the pioneer videoondemand company for both cable and broadband Internetmarkets.Taplinholdstwopatentsforvideoondemandtechnologies.ProfessorTaplin has provided consulting services on Broadband technology to the President of Portugal and theParliamentoftheSpanishstateofCatalonia.InMayof2010hewasappointedManaging DirectoroftheAnnenbergInnovationLab. Mr. Taplin graduated from Princeton University. He is a member of the Academy Of Motion PictureArtsandSciencesandsitsontheInternationalAdvisoryBoardoftheSingaporeMedia Authority and the Board of Directors of Public Knowledge. Mr. Taplin was appointed by GovernorArnoldSchwarzeneggertotheCaliforniaBroadbandTaskForceinJanuaryof2007. 9

FernandoPereira (InstitutoSuperiorTcnico,Lisboa/Portugal) Fernando Pereira is currently with the Electrical and Computers Engineering Department of Instituto Superior Tcnico and with Instituto de Telecomunicaes, Lisbon, Portugal(http://www.img.lx.it.pt/~fp/). HeisresponsiblefortheparticipationofISTinmanynational and international research projects. He acts often as project evaluatorandauditorforvariousorganizations. HeisanAreaEditoroftheSignalProcessing:ImageCommunicationJournal,amemberofthe Editorial Board of the Signal Processing Magazine, and is or has been an Associate Editor of IEEE Transactions of Circuits and Systems for Video Technology, IEEE Transactions on Image Processing,IEEETransactionsonMultimedia,andIEEESignalProcessingMagazine.Heisorhas been a member of the IEEE Signal Processing Society Technical Committees on Image, Video and Multidimensional Signal Processing, and Multimedia Signal Processing, and of the IEEE Circuits and Systems Society Technical Committees on Visual Signal Processing and Communications, and Multimedia Systems and Applications. He was an IEEE Distinguished Lecturerin2005andelectedasanIEEEFellowin2008. He is/has been a member of the Scientific and Program Committees of many international conferences. He has been the General Chair of the Picture Coding Symposium (PCS) in 2007 andtheTechnicalProgramCoChairoftheInt.ConferenceonImageProcessing(ICIP)in2010. He has been participating in the MPEG standardization activities, notably as the head of the Portuguesedelegation,chairmanoftheMPEGRequirementsGroup,andchairmanofmanyAd HocGroupsrelatedtotheMPEG4andMPEG7standards.HeisacoeditorofTheMPEG4 BookandTheMPEG21Bookwhicharereferencebooksintheirtopics. He won the first Portuguese IBM Scientific Award in 1990, an ISO award for Outstanding Technical Contribution for his contributions to the MPEG4 Visual Standard in 1998 and an HonourMentionoftheUTL/SantanderTottaAwardforElectrotechnicalEngineeringin2009. He has contributed more than 200 papers in international journals, conferences and workshops,andmadeseveraltensofinvitedtalksatconferencesandworkshops.Hisareasof interest are video analysis, coding, description and adaptation, and advanced multimedia services. 10

AlKovalick (AVID,U.S.A.) Al Kovalick has worked in the field of hybrid AV/IT systems for the past 18 years. Previously, he was a digital systems designer and technical strategist for HewlettPackard. Following HP, from 1999 to 2004, he was the CTO of the Broadcast Products Division at Pinnacle Systems. Currently, he is with Avid and serves as an Enterprise Strategist and Fellow. Alisanactivespeaker,educator,authorandparticipantwith industry bodies including SMPTE and AMWA. He has presentedover50papersatindustryconferencesworldwide andholds18USandforeignpatents.Heistheauthorofthe book Video Systems in an IT Environment; The Basics of Networked Media and FileBased Workflows (2nd edition, 2009). Al was awarded the SMPTE David Sarnoff Gold Medal in 2009. He has a BSEE degree from San Jose State University and MSEE degree from the UniversityofCaliforniaatBerkeley. HeisalifememberofTauBetaPi,IEEEmemberandaSMPTEFellow.

11

Demos

12

N-Screen Live Baseball Game Watching System: Novel Interaction Concepts within a Public SettingHogun Park, Geun Young Lee, Dongmahn Seo, Sun-Bum Youn, Suhyun Kim, Heedong KoImaging Media Research Center, Korea Institute of Science and Technology (KIST), Seoul, Korea {hogun, gylee, sarum, dmonkey, suhyun.kim, ko}@imrc.kist.re.kr

ABSTRACTRecently, as social media has taken place on an interactive TV domain, many researches have attempted to provide better emotional engagement and satisfaction. However, their approaches are still limited in utilizing many types of screens and supporting their social collaboration within a public setting. For example, when people are watching TV together, TV is not a suitable place to have a personal chat, and mobile phone is too small to support every sharing activity. In order to provide seamless social experience across any connected screens, in this paper, we present an N-Screen-based collaborative baseball watching system. It provides user engagement interfaces within a public setting and novel N-Screen interaction concepts.

this system, a public display like TV constitutes a novel global communication medium. It connects all surrounding displays to enable ubiquitous and cooperative watching. To address it, we have the following contributions: (1) User engagement interfaces within a public setting (2) New N-Screen interaction concepts and their implementation for baseball game watching. For evaluation, we implemented a live baseball game watching system on an android set-top box, an android mobile phone application, and a PC. In this paper, we present an overview of our proposed system (Section 2) and details of N-Screen-based watching system interfaces (Section 3).

Categories and Subject DescriptorsH.5.1 [Information Interfaces and Presentation]: Multimedia Information SystemsVideo; H.5.3 [Information Interfaces and Presentation]: Group and Organization Interfaces

2. SYSTEM OVERVIEW

General TermsDesign

KeywordsSocial TV, N-Screen, Live Sports Game Figure 1. Overall Framework of Proposed System

1. INTRODUCTIONAs social media research has taken place on an interactive TV domain, many researches have tried to provide more socialized experience on TV. However, most existing social TV platforms aim to connect viewers with their friends and families by providing a virtual shared space [1][2]. Even if their communication is more socialized around TV contents, they are still limited in utilizing many types of screens and supporting their social collaboration. In particular, a number of communities and corresponding individual viewpoints are derived from live events such as a baseball game and a musical performance. Depending on the interest of co-viewers, some parts of contents would be worthwhile to share, but others are not. To facilitate their social watching activity, it is necessary to provide seamless experience across screens of participants within a public setting. In this paper, we present a N-Screen collaborative sports watching system. In

The system framework of our proposed approach is illustrated in Figure 1. First, every user needs to register their personal displays like mobile phones to one of the nearest public displays. The public display serves as a medium to help group-based watching activities, and any users can initiate some sharing activities through the public display. In this system, it is newly introduced that one of participants can become a leader, so-called a media jockey (Section 3.2.2,) who can act as a media director and a producer. In our engagement interface, the media jockey takes a proactive role to organize all information and provide intermediate response to a live event and co-viewers. In other words, a group of users including a media jockey and viewers creates own social watching communities, and they collaboratively organizenew broadcasting stream. The community can be a group of friends/supporters and not necessarily placed on the same place. Our P2P streaming system supports to transmit/receive multiple live video streams, and media jockey plays a leading role of making broadcasting channel by organizing and selecting some of them. In our demonstration, a TV tuner, 3 HD cameras, and 3 spherical cameras were installed at a baseball stadium, and realtime streaming which guarantees synchronized [5] and low media zapping time [3] ( 0 as input and returns a solution with an approximation ratio of at most (1 + ) from the optimal, i.e. a solution which is worse than the optimal by at most a factor of (1 + ). The complexity of PTAS is polynomial in the instance size, but may be exponential in 1/ . We have shown that the problem is strongly NP-hard using a reduction from the 3-partition problem, which is strongly NP-hard [8]. This reduction holds even for instances where ad profits, ratings and frequency are equal to 1 and all ads can be allocated to all viewers, i.e. there is no personalization according to profiles. In addition, we have proven that personalization according to arbitrary profiles, i.e. arbitrary assignment restrictions, makes the problem APX-Hard, i.e. the problem has no PTAS unless P = N P . This proof was done using a reduction from the maximum 3-bounded 3-dimensional matching (3DM-3), which is APXhard as shown in [11], and it holds even for instances where ad lengths, profits, ratings and frequency are equal to 1. Although the problem has no PTAS, as described above, we developed PTASs for special instances of the problem: Instances of the Ads Allocation problem with no assignment restrictions and uniform ad lengths; Instances of the Ads Allocation problem with no as-

2.2

Multi-Period Uncertain Version

The multi-period uncertain version of the Ads Allocation problem is an extension of the deterministic case into a multi-period problem where the viewers viewing capacities are uncertain. While data regarding the ads requests, as well as data concerning the viewers profile (e.g. by asking the viewers), are known in advance, the data on a viewers viewing capacity is only a prediction of how much time a viewer will view TV within a certain period. Situations where viewers watch more or less time than expected are possible. The latter case, i.e. less viewing time than expected, is more problematic since there may be some ads that are not fulfilled which will cause a loss in revenue. However, the case of more viewing time than expected is also problematic, since knowing the actual viewing time in advance could result in allocation of more ads that in turn would increase revenue, which is our goal. We assume that each estimated viewing time, cj , of viewer vj is given together with some uncertainty factor 0 uj 1, where the real viewing capacity is a value in the range of [cj (1 uj ), cj (1 + uj )]. This model allows more realistic representation of the TV viewers, since their viewing capacity is not stable but can be estimated with a bounded error. The estimated viewing times can be based on viewing statistics. Such statistics are already available for some defined viewer groups. For example, according to BARB [3], in 2010 the average weekly viewing per person in Great Britain was 28:13. According to Nielsen [18], in 2009-2010 the average American watched 35:34 (hours/minutes) of TV per week,

signment restrictions, ad rating values that are taken from a constant number of options, and ad length val- ues that are a power of 2; Instances of the Ads Allocation problem with a bounded number of assignment restrictions, i.e. a constant num- ber of profiles, and uniform ad lengths. All of the PTASs we developed are based on generalizations of the PTAS for the multiple knapsack problem given in [5] and contain two steps: (1) selection of the ads to be assigned and (2) assigning the selected ads to viewers.

52

1 0.98 0.96 % Revenue 0.94 0.92 0.9 0.88 0.86 ProfitPerSec Backtrack Heuristic CPLEX 100 200 300 Number of Ads 400 500 % Revenue 100 75 50 Basic 25 CombinedHalf CombinedHalfLP Robust 0 10 20 30 40 50 % Uncertainty Factor

60

70

80

90

100

Figure 1: Algorithms performances for the deterministic version over instances of 100-500 ads.

Figure 2: Algorithms performances: average results for the multi-period uncertain version over instances of 1-8 periods. ments on different scenarios of uncertainty and number of periods, the performances of the heuristics were normalized to the performance in the deterministic case where the actual viewing capacities are known in advance. The heuristics we propose can be split into two categories: a robust heuristic which considers the worst case of viewing capacities and a modified rating heuristic which manipulates the ad ratings. Some of the interesting heuristics we propose adapt to the uncertainty and combine these two approaches. The C ombinedH alf heuristic considers the uncertainty of all periods in advance and allocates each ad to more viewers than needed, whereas the C ombinedH alf LP heuristic considers only the uncertainty of the last period. We use Basic to denote the performance of the heuristic which only adapts to the uncertainty and does not consider it in advance. As can be seen in figure 2, for higher uncertainty the performance of C ombinedH alf LP declines dramatically while the performance of the C ombinedH alf remains high, i.e. attaining above 85% of the revenue. Both algorithms obtain at least 87% of the revenue when the uncertainty is less than 60%, with some advantage to the C ombinedH alf LP heuristic. In general, it seems that the C ombinedH alf LP heuristic is preferred for lower uncertainty and the C ombinedH alf heuristic for higher uncertainty, where the break-even-point depends on the number of periods. For the full and detailed results of this problem version see [2].

3.2

Experimental Results

As a real world problem motivated by the industry, the focus of the research has been on developing algorithms that can be used and implemented (in contrast to pure theoretical research). Since the Ads Allocation problem is NP-Hard we developed heuristic algorithms which are common in solving instances of NP-Hard problems.

3.2.1

Deterministic Version Results

We developed several heuristic algorithms and evaluated them using simulations. Considering the size of the problem, i.e. thousands of ads and millions of viewers, we could not use an IP (Integer Programming) solver to solve the problem. In order to evaluate our heuristics we reduced the problem instances size and compared the results to those of a state-of-the-art IP solver, i.e. IBM ILOG CPLEX [10]. For the tested instances our heuristics returned results, within a few seconds, which were very close to the upper bound of the optimal value given by the solver (for some instances we could not find the optimal solution using CPLEX even with- out a runtime limit). Since realistic problem instances are much more, e.g. millions of viewers, with which CPLEX is unable to deal, the heuristic solutions, displaying very good performance, seem to be a good solution. The instances we tested include different ratios of ads per viewers, and different kinds of ad profiles, e.g. general ads that are relevant to all viewers vs. specific ads relevant to a unique target population. The heuristics we propose can be split into three categories: payment oriented, target population oriented and backtrack oriented. One of the interesting heuristics we propose is the BacktrackH euristic algorithm which takes into account the personalization level and payment of the ads in addition to the backtrack process. Another interesting heuristic is the P rof itP erSec algorithm, a greedy heuristic that prefers ads with a high profit per sec. Performance results of these algorithms are presented in figure 1. Our BacktrackH euristic outperforms the other heuristics and the IP solver, denoted as C P LEX , and on average attains 98% of the possible revenue. For the full and detailed results for this problem version see [1].

4.

RESEARCH PLAN

During the current year and the upcoming fourth year of research we plan to continue to investigate the problem, develop new algorithms and extend the evaluations. The detailed research plan, described below, includes completed, current and future work. Develop heuristics for the deterministic version of the Ads Allocation problem, and evaluate and compare them to the CPLEX IP solver. Completed and pub- lished (see [1]). Develop heuristics for the multi-period uncertain ver- sion of the Ads Allocation problem and Completed and published (Best winner, see [2]). evaluate them. paper award

3.2.2

Multi-Period Uncertain Version Results

This version of the problem naturally falls into the cat- egory of a multi-period problem where after each period, when some of the uncertainty has been revealed, the ads can be reallocated. Therefore, we present a sequential solu- tion procedure for the problem and propose several heuristic algorithms for solving it. Through computational experi-

Develop approximation algorithms for the determinis- tic version of the Ads Allocation problem. Such results provide bounds on the attainable revenue resulting in

53

the guaranteed quality of our heuristics. We already have several theoretical results including hardness results and polynomial time approximation schemes for special instances of the problem (see Section 3.1). Current work, partially completed. Extend the evaluation of the multi-period uncertain solutions for the Ads Allocation problem under different types of data environments, for instance, by altering the number of ads, the number of viewers, the uncertainty factors, the number of periods, etc. In addition, collect real data regarding viewers viewing capacities and evaluate the solutions using this data. Current work. Address the Ads Allocation problem under relaxation of the all-or-nothing rating and frequency constraints. The all-or-nothing constraints, e.g. the request to allocate the ad to the exact number of required different viewers, seem to have a tremendous effect on the problem while in reality minor violations can be ignored. Current work. Consider other special constraints of interactive TV, such as past interactions with viewers, current watched content, user interactive limitations (e.g. problematic back-channel for participation TV or television commerce services), etc. Future work.

5.

RESEARCH CONTRIBUTION

Since the aim of this research is to develop new algorithms to allow optimized ad personalization in the interactive TV environment and other enhanced TV mediums, its contribution to the interactive TV industry is consequential. This research will allow the industry to maximize revenues from advertising as well as deliver more relevant and interesting advertisements to the viewers. While other studies focus on selecting ads most suitable to the viewers, this research focuses on optimizing such allocations given the suitable ads for each viewer. As far as we know there is no other work underway which presents solutions to this problem while taking into account all the special constraints of the TV industry. Along with its contribution to the industry, this work can also be relevant to other domains and industries faced with similar assignment problems, e.g. packing of containers with multi all-or-nothing constraints. In addition to our heuristics algorithms we also present theoretical work which contributes to the theoretical investigation of the General Assignment Problem (GAP), the Multiple Knapsack Problem (MKP), and the Multiple Knapsack problem with Assignment Restrictions (MKAR).

6.

REFERENCES

[1] R. Adany, S. Kraus, and F. Ordonez. Personal Advertisement Allocation for Mobile TV. In International Conference on Advances in Mobile Computing & Multimedia, 2009. [2] R. Adany, S. Kraus, and F. Ordonez. Uncertain Personal Advertisement Allocation for Mobile TV. In International Conference on Advances in Mobile Computing & Multimedia, 2010. [3] BARB. Barb reports: Monthly total viewing summary. http://www.barb.co.uk, 2010.

[4] T. Bozios, G. Lekakos, V. Skoularidou, and K. Chorianopoulos. Advanced Techniques for Personalized Advertising in a Digital TV Environment: The iMEDIA System. In Proceedings of the eBusiness and eWork Conference, pages 10251031, Venice, Italy, 2001. [5] C. Chekuri and S. Khanna. A PTAS for the multiple knapsack problem. In Proceedings of the eleventh annual ACM-SIAM symposium on Discrete algorithms, pages 213222. Society for Industrial and Applied Mathematics Philadelphia, PA, USA, 2000. [6] M. Dawande, J. Kalagnanam, P. Keskinocak, F. Salman, and R. Ravi. Approximation Algorithms for the Multiple Knapsack Problem with Assignment Restrictions. Journal of Combinatorial Optimization, 4(2):171186, 2000. [7] V. Dureau. Addressable advertising on digital television. In Proceedings of the 2nd European conference on interactive television: enhancing the experience, Brighton, UK, MarchApril 2004. [8] M. R. Garey and D. S. Johnson. Computers and In-tractability. A Guide to the Theory of NP-Completeness. W.H. Freeman, New York, 1979. [9] Google AdWards. http://adwords.google.com. [10] IBM ILOG CPLEX Optimizer. http://ibm.com. [11] V. Kann. Maximum bounded 3-dimensional matching is MAX SNP-complete. Information Processing Letters, 37(1):2735, 1991. [12] G. Kastidou and R. Cohen. An approach for delivering personalized ads in interactive TV customized to both users and advertisers. In Proceedings of European conference on interactive television, 2006. [13] E. M. Kim and S. S. Wildman. A deeper look at the economics of advertiser support for television: the implications of consumption-differentiated viewers and ad addressability. Journal of Media Economics, 19:5579, 2006. [14] O. E. Kundakcioglu and S. Alizamir. Generalized assignment problem. In C. A. Floudas and P. M. Pardalos, editors, Encyclopedia of Optimization, pages 11531162. Springer, 2009. [15] G. Lekakos and G. Giaglis. A Lifestyle-Based Approach for Delivering Personalized Advertisements in Digital Interactive Television. Journal of Computer-Mediated Communication, 9(2):0000, 2004. [16] M. Lopez-Nores, J. Pazos-Arias, J. Garc a-Duque, Y. Blanco-Fernandez, M. Mart n-Vicente, A. Fernandez-Vilas, M. Ramos-Cabrer, and A. Gil-Solla. MiSPOT: dynamic product placement for digital TV through MPEG-4 processing and semantic reasoning. Knowledge and Information Systems, 22(1):101128, 2010. [17] Negev consortium. http://www.negevinitiative.org. [18] Nielsen Media Research. Snapshot of television use in the u.s. http://nielsen.com, September 2010. [19] Z. Nutov, I. Beniaminy, and R. Yuster. A (11/e)-approximation algorithm for the generalized assignment problem. Operations Research Letters, 34(3):283288, 2006. [20] SintecMedia. http://sintecmedia.com.

54

Video access and interaction based on emotionsEva OliveiraLaSIGE, University of Lisbon FCUL, 1749-016 Lisbon, Portugal, IPCA, 4750-117 Arcozelo BCL, Portugal +351 2175000533

eoliveira@ipca.pt

ABSTRACTFilms are by excellence the form of art that exploits our affective, perceptual and intellectual activity. Technological developments and the trends for media convergence are turning video into a dominant and pervasive medium and online video is becoming a growing entertainment activity on the web and iTV. Alongside, Human Computer Interaction research community has been using physiologic, brain and behavior measures to study possible ways to identify and use emotions in human-machine interactions. In our work we explore emotional recognition and classification mechanisms in order to provide video classification based on emotions, and to identify each users emotional states so as to provide different access mechanisms. We also focus on emotional movie access and exploration mechanisms to explore ways to access and visualize videos based on their emotional properties and users emotions and profiles.

of watching movies easy and doable. Films are by excellence the form of art that evolves affective, perceptual and intellectual activity. It is called as a way to transport us to new worlds, lives and fantasies by telling stories [11]. By combining diverse symbol systems, such as pictures, texts, music and narration, video is a very rich media type, often engaging the viewer cognitively and emotionally, and having a great potential in the promotion of emotional experiences. It has been used in different contexts: as a way to capture and show real events, to create and visualize scenarios not observable in reality, to inform, to learn, to tell stories and entertain, and to motivate; as movies, documentaries or short video clips. Isen et. al. [9] attested this potential, when she and her colleagues experimented the effect of positive affect in her patients, inducted by ten-minu-te comedy films. The study of films as an emotion induction method has some reports dated from 1996, as [12] reported, analyzing the mental operations of film viewers and discussing how emotions guide the motivation of perception and consequently the control of our attention by cinematographic narratives. Emotion studies have been done over the last few years, since it became proved that they are fundamental in cognitive and creative processes. In fact, understanding emotions is crucial to understanding motivation, attention or aesthetic phenomena. There is an increasing awareness in the HCI community of the important role of emotion in human computer interactions and interface design, and new mechanisms for the development of interfaces that register and respond to emotions have been studied [3]. Gathering emotional information from users can contribute to create emotional context in applications interfaces. Rosalind Picard in [17] defends that systems that ignore the emotional component of human life are inevitably inferior and incomplete, and she states that systems that provide a proper and useful social and emotional interaction are not science fiction but a science fact. Societys relation with technology is changing in such ways that it is predictable that, in the next years, Human Computer Interaction (HCI) will be dealing with users and computers that can be anywhere, and at anytime, and this changes interaction perspectives for the future. Human body changes, expressions or emotions would constitute factors that became naturally included in the design of human computer interactions [2]. There is a wide spectrum of areas that investigate emotions with different, but complementary, perspectives. For example, in the neurobiological area, [5] showed that emotions play a major role on cognitive and

Categories and Subject DescriptorsH.5.1 [Information Interfaces and Presentation (I.7)]: Multimedia Information Systems video; H.5.2 [Information Interfaces and Presentation (I.7)]: User Interfaces screen design;

General TermsDesign, Experimentation, Human Factors.

KeywordsAffective computing, Emotion-aware systems, Human-centred design, Psychophysiological measures, Pattern-recognition, Discriminant analysis, Support vector machine classifiers, Movies classification and recommendation.

1.INTRODUCTIONVideo growth over the Internet changed the way users search, browse and view video content. Watching movies over the Internet is increasing and becoming a pastime. The possibility of streaming to TV Internet content, advances in video compression techniques and video streaming have turned this recent modality

55

decision making processes; HCI aims to understand the way users experience interactions and strives to stimulate the sense of pleasure and satisfaction by developing systems that focus on new intelligent ways to react to user's emotions [10]. HCI is also concerned with evaluation and usability, which includes evaluating the extent and accessibility of the systems user interface, accessing a users experience in interaction and identifying specific problems. The advent of rich multimedia interfaces has been providing new technological foundations to support these emotional trends in HCI. Currently, affective computing systems are being developed that are able to recognize and respond to emotions with the aim of improving the quality of human-computer interaction. Part of this research has concentrated on solving many technical issues involved with emotion recognition technologies. For example, [14] describe their work with sensors in the context of a study on emotional physiological response. According to [1] physiological measures such as galvanic skin response or pupil dilation constitute objective factors but are not easily correlated to particular emotions. Moreover, there are variations in rates which are due to normal individual differences among users, and intrusive wires or sensors may affect users behaviors. To circumvent this, less intrusive devices were developed [17]. In our work we are developing a novel emotional recognition approach based on pattern recognition techniques, based on discriminant analysis and support vector machine classifiers, which are validated using movies scenes selected to induce emotions ranging from the positive to the negative valence dimension, including happiness, anger, disgust, sadness, and fear. We present the system, iFelt, an interactive web video system designed to learn users emotional patterns, create emotional profiles for both users and videos, and explore this information to create emotion based interactions. The iFelt system is composed of two components. The Emotional Recognition and Classification component performs emotional recognition and classification and semantic representation of emotions in order to provide video classification based on emotions, and to identify each users emotional states so as to provide different search and access mechanisms. The Emotional Movie Access and Exploration component explores ways to access, to search, to represent and visualize videos based on their emotional properties and users emotions and profiles.

Considering the effect of the emotions in a persons attention, motivation and behavior, a scenario where it would be beneficial to have emotional impact videos to capture viewers attention is in educational contexts, where video could capture students attention in different ways, either to focus or to relax. The induction of emotions using movies has been largely used in psychology studies [8] and in health related studies. In fact, experimental studies confirmed that positive emotions can have a beneficial effect on physical health [15]. The development of new mechanisms to catalog, find and access movies based on emotions could help to assess videos emotion impact, and to find movies or scenes that tend to induce a certain feeling in the users. It could also aid filmmakers to perceive the emotional impact of their movies and, in particular, the emotional impact of each scene and compare it to the intention they had for the scene impact, and relate it to the adoption of specific special effects, acting approaches and settings. Moreover, actors may also be able to perceive their impact in a specific act. Finally, movie consumers may be able to explore movies by the emotions stirred by the content in multiple ways, compare their emotional reactions with other users reactions and see how they change overtime. Other challenges in accessing video is the fact that it conveys a huge amount of audiovisual information that is not structured and that changes along time, and so, accessing all the data that a video can provide is often not an easy task. Semantic descriptors, like its emotional properties, either expressed on the movie or felt by the users, can be used to tag some information of the video. And once this information is collected, we can try to use it for a better and meaningful organization of the individual and collective video spaces, to search, and even to provide new forms of visualization and interaction [7,18]. Visualization techniques, emerged from research rooted primarily on visual perception and cognition [4], can actually help to handle the complexity and express the richness in these information spaces. Video visualization can be an intuitive and effective way to convey meaningful information in video [18]. These issues can be synthesized in the following problem statement addressed in this work: Emotional classification based on physiologic information acquired from users when watching films, improves the relevance of movie search retrieval, contributes to enrich movie recommender systems and enables the design of emotional aware user interfaces for movie visualization by adapting their structure and their visualization elements and tools.

2.PROBLEMEmotions can be expressed in a variety of ways such as body expressions (facial, vocals, body posture), or neurophysiologic symptoms (respiration, heart-rate, galvanic skin response and blood pressure). Accordingly, the Human Computer Interaction research community has been using physiologic, brain and behavior measures to study possible ways to identify and use emotions in human-machine interactions [10]. However, there are still challenges in the recognition processes, regarding the effectiveness of the mechanisms used to induce emotions. The induction is the process through which people are guided to feel one or more specific emotions, which provokes body reactions. Some relevant works showed that films were one of the most effective ways to induce emotions. J. Gross et al. [6] tried to find as many films as possible to elicit discrete emotions and find the best films for each discrete emotion. In 1996, a research group [12] tested eleven induction methods and concluded that films are the best method to elicit emotions (both positive and negative). The exploration of movies by their emotional dimensions can be used for entertainment, education or even medical purposes.

3. PHD OBJECTIVESIn order to address the issues identified in the problem statement presented above, and focusing on the research questions that emerged, four main goals were defined for this thesis, more specifically to: Improve movie search mechanisms, making information retrieval more relevant through the use of emotional profiles from users; Enrich recommender systems by adding emotional information allowing movie suggestions based upon emotional profiles of users and movies emotional profiles; Access and visualize characteristics of videos; videos based on emotional

Adapt user interface aspects with emotional awareness features that allow controlling the movie sequence

56

visualization, the complementary information available and even the way in which the collected information is displayed; To address these objectives we are going to follow a methodology described in the next section.

4.METHODOLOGY AND PHD CONTRIBUTIONSThe methodology used to develop this work was the following. First, an extensive research literature review was performed so as to understand the role of emotions in the context of affective computing with the precise objective of identifying the importance of emotional approaches in Human Computer Interaction, on Multimedia Information Retrieval and in Video Processing, along with the clarification of associated problems and limitations. Background literature on emotional theories, emotion recognition, biosignal processing, classification techniques, video analysis and low-level feature extraction, emotional design and recommendation systems was also covered with the objective of providing a framework of key concepts and technologies on which to base the design of a system architecture that addresses the emotional classification of users and movies as well as its representation and access. We developed an interactive web video application - iFelt - developed to learn users emotional patterns using movies scenes selected to induce emotions. - The iFelt system has 2 main components aimed at: emotional movie Content Classification and emotional movie Access and Exploration. This last component aims to provide video access and visualization based on their emotional properties and users emotions and profiles. We are designing different methods to access and watch the movies, at the levels of the whole movie collection, and the individual movies. The first prototype is focused towards the access based on the emotions felted by the user, to explore and evaluate emotional paradigm, on top of which we will later add the other perspectives. The design options are thoroughly addressed in [13]. Next we present our main contributions so far based on the problems stated before.

watching movie scenes to create an engine to support user interaction, and to enhance automatic recognition of users emotional states. We selected a set of movie scenes to induct subjects to feel five basic emotions (happiness, sadness, anger, fear and disgust) and the neutral one. Every subject watched 16 scenes (four of happiness, four of sadness, four of fear, two of disgust and two of anger) and one neutral scene. Based on their feedback, we associated the captured physiological signals with emotional labels, and trained our engine. Eight movies were randomly chosen from the total pool of 30. An average of two subjects watched these eight movies, and were classified by the system. With the SVM classifier, the overall average recognition rate is 69% (s.d. 5.0%), which represents a 49% improvement over random choice, whereas the k-NN classifier produced an overall average recognition rate of 47% (s.d. 9.3%). The SVM classification score shows promise that the iFelt system can be used to automatically evaluate human emotions.

4.2.Emotional Movie Access and ExplorationiFelt is an interactive web video application that allows to catalog, access, explore and visualize emotional information about movies. It is being designed to explore the affective dimensions of movies in terms of their properties and in accordance with users emotional profiles, choices and states. Although iFelt supports any kind of video, we are focusing our analysis in movies. The iFelt system has two main goals: 1) Emotional Movie content classification: to provide video classification based on emotions, either expressed in the movies, or felt by the users; 2) Emotional Movie access and exploration: to access and visualize videos based on their emotional properties and users emotions and profiles. In iFelt, we created different levels to access and explore movies: the 1) movies space where users get a view over the movies existing in the system, with information about their dominant emotions. We designed different representations, including movie lists and emotional wheels, where the movies are represented by a colored circle, with their dominant emotion color, in ways that represent the level of emotion dominance in each movie.; the 2) emotional scenes space, where users can obtain a view of the scenes of the movies based on the scenes dominant emotions, and allowing for e.g. to access the individual movies, but presenting only the scenes with the selected emotion, as emotional summaries of the movies; the 3) individual movie level, where the movie can be watched and, in addition, information about its dominant emotions and emotional scenes can be viewed, for e.g. through an emotional timeline that represents the emotional scenes along the movies; and 4) users have an emotional profile, with emotional information about their movies, that is movies classified from their own perspective or view, and statistical information concerning their history, in terms of the emotional classification of movies they watched. The Emotional Movie Access and Exploration are thoroughly described in [13].

4.1.Emotional Recognition and ClassificationOur emotional recognition and classification component is grounded in the induction of emotional states by having users watch movie scenes. The recognition process included two important phases, the training and the testing phases. Inspired by the works of [16], we based our testing phase in an induction of emotion using a set of emotional movie scenes. This component can be divided into two main modules, the Biosignal Recording module and the Pattern Recognition module. Biosignal recording uses biosensors for measuring Galvanic Skin Response (GSR), Respiration (Resp) and Electrocardiogram (ECG) and is responsible for users biosignals recording and signal processing pipeline. These sensors were specifically chosen as they record the physiological responses of emotion, as controlled by the autonomous nervous system. The Pattern Recognition module uses discriminant analysis, support vector machine and K-nn classifiers to analyze the physiological data and it was validated by the usage of specific movie scenes selected to induce particular emotions. Our objective was to determine whether our classification engine is sufficiently accurate to automatically recognize emotional patterns from new data with a reasonable success rate. Another goal was to determine if the selected scenes had the same emotional impact in all the users in order to measure the importance of the scene for eliciting a specific emotion. Eight participants, averaged 34 year, were submitted to the experiment. In our study we are using the subjects data obtained while

5.FUTURE WORK AND PHD JUSTIFICATIONOur ongoing research intends to support real-time classification of discrete emotional states from biosignals for multimedia content classification and user interaction mechanisms by developing emotional aware applications that react in accordance to user's emotions. We are considering using emotion

57

recognition to automatically create emotional scenes, recommend movies based on the emotional state of the user and adjust interfaces according to users emotions and based on emotional regulation theories. By creating emotional profiles for both movies and users, we are developing new ways of discovery interesting emotional information in unknown or unseen movies, compare reactions to the same movies among other users, compare directors intentions with users effective impact, analyze over time our reactions or directors tendencies. Regarding visual exploration and access mechanisms of emotional information the next step would be to improve and extend the system in accordance with users feedback, our own evaluation of the current design and implementation, and some of the ideas we originally had and that were not yet included in the current version. Some of the future features include: extending the concept of video summaries to present movies in chosen emotional perspectives and preferences, with more criteria other then selecting scenes with one chosen emotion; summarizing or searching or recommending movies based on users current emotional states, or defined emotional criteria; to find movies by example, i.e. with emotional timelines similar to the timeline of a given movie; exploring the visual representation of huge amounts of movies and extend selecting and browsing methods based on more sophisticated and powerful filters and searches; and to include support for historical emotional information gathered along time, so we can witness the evolution of users emotional reactions to movies over time, and compare it to other perspectives, including the actors and directors involved, in the several movies genres. We also intend to make all this information more available, or visible, on the web as a shared and recommender environment based on the emotional classification of movies, useful for the general public, as well as for more professional perspectives of directors and actors. Finally, iFelt is currently focused in movies and the web environment, but this same approach can be useful and interesting to be explored with other types of videos, as is the case of advertisement videos that typically aim at specific emotional reactions from the viewers; and from interactive TV and video on demand services. The core functional and interface features could be the same, but some new requirements in these contexts might involve some adaptations or extensions.

[4]

Card, S.K., Mackinlay J.D., and Shneiderman, B. 1999 Readings in Information Visualization: Using Vision to Think, San Francisco, California: Morgan-Kaufmann. Damasio, A. (1995). Descartes' Error. Harper Perennial; Gross, J. J. & Levenson, R. W. (1995). Emotion elicitation using films. Cognition & Emotion, 9(1), 87-108. 987-108 Hauptmann, A. G. 2005. Lessons for the Future from a Decade of Informedia Video Analysis Research. Int. Conf. on Image and Video Retrieval, National Univ. of Singapore, Singapure, July 20-22. LNCS, vol 3568, pp.1-10, Aug. Huppert, F. 2006. Positive emotions and cognition: developmental, neuroscience and health perspectives. In Forgas J.P. (Ed.), Hearts and minds: Affective influences on social cognition and behavior., Psychology Press, New York. Isen A. M., Daubman K. A., and Nowicki G. P. (1987) Positive affect facilitates creative problem solving. Journal of personality and social psychology, 52:112231.

[5] [6] [7]

[8]

[9]

[10] Maaoui, C., Pruski, A., & Abdat, F. 2008. Emotion recognition for human-machine communication. 2008 IEEERSJ International Conference on Intelligent Robots and Systems, 1210-1215. [11] Metz, C. & Taylor, M. 1991. Film language: A semiotics of the cinema. University of Chicago Press. [12] Mnsterberg, H. (1970) The film: A psychological study: The silent photoplay in 1916. Dover Public. [13] Oliveira, E., Martins, P., Chambel, T. 2011. iFelt Accessing Movies Through our Emotions. In Proceedings of EuroITV'2011, 9th. European Conference on Interactive TV and Video, ACM SIGWEB, SIGMM & SIGCHI, Lisbon, Portugal, Jun 29-1Jul, 2011. [14] Peter, C., & Herbon, A. (2006). Emotion representation and physiology assignments in digital systems. Interacting With Computers: 18 (2), 139-170. [15] Philippot, P., Baeyens, C., Douilliez, C., & Francart, B. (2004) Cognitive regulation of emotion: Application to clinical disorders. In P. Philippot & R.S. Feldman (Eds.). The regulation of emotion. New York: Laurence Erlbaum Associates. [16] Picard, R. W., Vyzas, E., & Healey, J. 2001. Toward Machine Emotional Intelligence: Analysis of Affective Physiological State. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1175-1191.1175-1191 [17] Picard, R.W., 1997. Affective Computing. MIT Press, Cambridge, MA. [18] Teresa Chambel, Telmo Rocha and Joo Martinho, "Creative Visualization and Exploration of Video Spaces". In Proceedings of Artech'2010 : Envisioning Digital Spaces, 5th International Conference on Digital Arts, pp.1-10, Guimares, Portugal, Apr 22-23, 2010.

6.REFERENCES[1] Axelrod, L., Hone, K.S. Affectemes and all affects: a novel approach to coding user emotional expression during interactive experiences. Behaviour & Information Technology,25, 2, March-April, 159-173 (2006) Being Human: Human-Computer Interaction in the Year 2020. http://research.microsoft.com/hci2020/, 2007. Brava, S., Nass, C., & Hutchinson, K., 2005. Computers that care: investigating the effects of orientation of emotion exhibited by an embodied computer agent. International Journal of Human-Computer Studies, 62(2), 161-178.

[2] [3]

58

Posters

59

Online iTV use by older people: preliminary findings of a rapid ethnographical study*Susan Ferreira, **Sergio Sayago, *Valeria Righi, *Guiller Maln, *Josep Blat*Interactive Technologies Group, Universitat Pompeu Fabra RocBoronat,138 (Barcelona, Spain)

(susanferreira, righi.vale, gmalon)@gmail.com, josepblat@upf.es**Digital Media Access Group, School of Computing, University of Dundee, DD1 4HN (Dundee, Scotland)

sergiosayago@computing.dundee.ac.uk

ABSTRACTThis poster presents preliminary findings of a rapid ethnographical study of online iTV use by some 40 older people during 2 months. Whereas some research has addressed iTV accessibility for older people, online iTV has largely been overlooked. This paper presents some key issues of online iTV by ordinary older people who are motivated to technology uptake, their attitudes towards interacting with iTV on the traditional TV and a number of issues related to ICT use which can inspire the design of more enriching and inclusive online iTV services. on these opportunities if online iTV is not accessible to them. Most of todays older people have a lack of experience with digital technologies. Furthermore, these technologies have largely been designed without taking older people into account. Thus, there is a need to further online iTV research with older people. A number of previous studies have addressed iTV accessibility with older people (0, 0, 0). These have focused on developing novel and interesting software prototypes. For instance, 0 describes a system aimed at sharing multimedia content and 0 discusses a tool intended to give support to communication. Other studies have explored through interviews the reasons why iTV is unappealing to older people 0. Whereas the interaction barriers (e.g. difficulties using the mouse, understanding computer jargon) that older people face while using other digital technologies have been explored before (0,0), very little is known about those they encounter when interacting with iTV. 0 points out that significant work still should be done to design interfaces that are more usable to older people and support better their skills and abilities. Moreover, none of the studies reviewed above has addressed online iTV, despite the proliferation of it. Examples are the BBC iPlayer of the BBC in the UK 0 and TV3 A la Carta of TVC 0 in Catalonia. There is also a lack of information about how older people use (or would use) online iTV in out-oflaboratory conditions. Most of the studies reviewed above have been conducted in laboratory conditions, which concurs with the main approach adopted in HCI research with older people to date 0. Yet, there is growing awareness in HCI that understanding interactions as they happen in everyday environments is a crucial element to designing better, and therefore, more accessible interactions 0. An extended ethnographical study of technology use by older people revealed, for instance, that accessibility issues due to cognition limit older peoples interactions with digital technologies in out-of-laboratory conditions more seriously than those due to vision 0. This paper presents the preliminary results of a rapid ethnographical study 0 of iTV use by older people. This work is being conducted within Life 2.0 0, a research project aimed at making the network of social interactions more visible to older people. This will be done by providing them with an accessible platform consisting of collaborative ICT that track and locate relevant members of their social networks (i.e. relatives, friends and caregivers). The Life 2.0 platform will allow older people

Categories and Subject DescriptorsH.5.2 [Information Interfaces and Presentation]: User Interfaces - User-centered design; K.4.2 [Computers and Society]: Social Issues - Assistive technologies for persons with disabilities;

General TermsDesign, Human Factors.

KeywordsiTV, older people, accessibility, ethnography

1. INTRODUCTIONThe traditional model of watching TV is changing. Today, people can interact with and create TV content almost anywhere. Online iTV (interactive TV on the web) allows us to share and produce content, reinforce communication and personalize information and services. We argue in this paper that exploring online iTV with older people (60+) is worthwhile. Online iTV services can and should be useful to older people, who consume a lot of TV. However, and despite an increasing ageing population, they run the risk of missing out

60

and their social networks to communicate amongst themselves through phone calls, text messages, advanced multimedia content distribution systems (e.g. IPTV, interactive digital signage and WebTV) and video telephony/conference solutions. The preliminary results indicate that our participants are interested in online iTV, especially in re-watching their favorite programs and watching those TV programs they missed. They are also keen to recommend TV programs to important members of their social circles. Interestingly, our participants do not have any interest in writing comments related to TV programs, despite the popularity of the comments in online iTV and, in general, Web 2.0. The results also indicate some interaction and social issues that should be considered in the design and evaluation of future iTV services, namely, privacy and social exclusion. The remainder of the paper is organized as follows. Section 2 describes the rapid ethnographical study. Section 3 presents the initial results of this study. Section 4 discusses the results and the research approach. Section 5 describes our ongoing and future research activities.

Table 1 - Ongoing fieldworkActivity Workshop on Google Maps Description Hands-on introduction to Google Map, collaborative map. Hands-on introduction to blogs, creation of a blog with blogger. Hands-on introduction to TV channels video on demand Web Page and Youtube. Hands-on introduction to Facebook. Discussion about collaborative maps and blogs. Download and edit pictures from the web about gardens. Create/share documents (e.g. calendar, power point). Idem Technology Google Maps Participants 12 (6 men / women) Duration 2 sessions 2-hour session week 2 sessions 2-hour session week 1 session 2-hour / /

Workshop on weblogs

Blogger

11 (5 men / 6 women)

Workshop TV on the Web

Internet Explorer, Mozilla Firefox

11 (6 men / 5 women)

Workshop Facebook Participatory Design Workshop

Facebook

9 (6 men / 3 women) 10 (5 men / women)

1 session 2-hour 1 session 2-hour

2. RAPID ETHNOGRAPHICAL STUDY 2.1. Context: goraWe have conducted this study in gora, a 20-year-old association in Barcelona, which intends to integrate into Catalan society people who are, or might be, excluded from it, e.g. immigrants, non-educated and older people. gora considers that mastering digital technologies is a crucial aspect in achieving this inclusion. Thus, courses in computing, Internet access and workshops are provided. These and other activities are free of charge and participants, which is the term used by gora to reinforce the inclusion aspect of their work, decide what technologies they want to (learn to) use. This decision is often grounded in the participants daily needs or interests.

Map paper prototype and some other images and text. Internet Explorer, Mozilla Firefox, MS office tools, pictureediting tools, e-mail. Idem

Course on Gardens of the World

9 (4 men / women) 11 (6 men / women) 9 (4 men / women) 9 (4 men / women)

5 5 5 5

4 sessions 2-hour session week /

Course on Wild life and Nature

13 (4 men / 9 women) 11 (4 men / 7 women)

2 sessions 2-hour session / week

2.2.

Participants, iTV and research methods

We have conducted 27 hours of fieldwork over a 2-month period. The fieldwork activities consisted of in-situ observations of and informal conversations with around 40 older people (aged 60-75) while using several digital technologies. We ran 5 workshops, in which we explored technologies relevant for Life 2.0, such as and online iTV (YouTube and ondemand Spanish and Catalan TV channels), Google Maps, weblogs and Facebook. We also participated in 2 courses, which were organized by gora as part of their activities to foster the use of digital technologies amongst the older population and that had no specific connection with Life 2.0, in order to develop a more comprehensive understanding of older peoples interactions with digital technologies. The fieldwork was conducted in the goras computer room. Our participants can be considered as a heterogeneous user group. They originated from different Spanish and Catalan regions, and had different educational levels (ranging from primary to secondary school). In terms of computer skills, 27 were familiar with basic and more advanced aspects of interacting with computers, such as when to left- or right-click and look for information online. Table 1 summarizes the fieldwork activities.

We have recorded fieldnotes by using paper and pencil, and photographs. Our participants wrote down their notes by using notebooks and were used to other people in gora doing the same. Thus, laptops and video cameras were intrusive. Also, there are no laptops in the goras computers room, and our participants are not used to being videoed during their everyday interactions with them. We have analyzed the fieldnotes by using Grounded Theory 0, i.e. while gathering the data. We have conducted initial, axial and selective coding. We discuss the initial results next2.

3. PREMILINARY RESULTSWe first discuss some aspects of how older people use or would use online iTV. We then deal with their attitudes towards watching online iTV on the traditional TV. We also address other aspects related to interacting with online iTV, such as privacy and social inclusion, which emerged from the analysis and we consider crucial in better understanding and designing online iTV services with older people.

3.1. Using online iTV2

0 analyses the data gathered in this ethnographical study in terms of the potential of geo-positioning services based on ICT for social inclusion amidst older people and their social circles.

61

Whereas none of our participants had used online iTV before, all of them were eager to use it. They showed interest in sharing videos with people they did know, namely, their children, grandchildren and close friends. They are also key actors in the use of e-mail by older people 0. Our participants were also interested in the possibility of either re-watching TV programs or watching those they missed. We observed that the participants were keen to share videos by e-mail. This is probably because all of them send and receive emails. However, no participant was interested in either commenting or rating videos. We found a similar result in a previous study of YouTube we conducted with another group of older people 0. In both studies, older people reported not being interested in the opinions of other people, and their strategy for commenting and rating is likely to be sending an e-mail to their children, grandchildren and close friends. Part of our future work is to explore this issue in detail. We observed, and participants reported, that websites such as TVE a la carta 0, TV3 a la carta 0 and BTV a la carta 0, were fairly easy to use. Despite the considerable amount of information presented on these sites, each participant searched for his or her favorite TV programs independently (i.e. without relying on us). They were much more dependent (i.e. relying on us) to conduct other activities, such as downloading an attachment received in an e-mail, in which we consider they deal with much less amount of information. Several factors might account for this interesting finding, which highlights the relevance of cognition in agreement with 0 such as the familiarity with the task at hand or the desktop (difficult to understand) and online iTV metaphors (similar to TV magazines). We will explore this result further in our future research.

Concurring with previous studies of ICT and ageing (e.g. 0, 0, [13]), the family is very important in the use our participants make of online iTV services. This suggests that an online iTV channel with my relatives, or other forms of communication with them mediated by online iTV, can encourage the uptake and use of it by older people, as well as reducing social exclusion. All our participants considered that using digital technologies was crucial in being included in current society. Thus, although numerous older people are not motivated to use ICT, the effort our participants make to use them should be understood as an opportunity to design better iTV services for them (and all of us).

4. DISCUSSIONWe have addressed online iTV with a heterogeneous group of older people in an attempt to improve current understanding of iTV with them and their interactions with digital technologies. The preliminary results are rich, as they have shown expected and unexpected findings and dealt with a broad number of issues. For instance, whereas re-watching favorite TV programs can hold true for the use of online iTV by other user groups, our participants seem to have their own strategy for rating and commenting online iTV content, and this strategy is not related to writing comments or clicking on I like it on the website. We have also identified their changing and positive attitudes towards iTV, and the importance of privacy, the family and social inclusion in understanding their interactions with iTV (and other technologies) and designing better ones for them. We have adopted a research approach which has seldom been used in previous studies of iTV (and HCI in general) with older people: rapid ethnography 0. Although the results are preliminary, our first experiences of recording in-situ observations of and conversations with the participants while using the technology in out-of-laboratory conditions suggest that the method has great potential to further our understanding of older people as iTV users. Whereas it is common to include extracts of fieldnotes in ethnographical studies, we have not included any because we feel much more research is needed to expand, confirm and reject our initial ideas. At this stage of our research, we have not made any comparison between younger and older peoples interactions with online iTV because we consider we need to understand much deeply the current gap in iTV research with older people and work with more participants in order to make valid, significant and useful comparisons. Finally, let us also note that whereas the number of participants who took part in activities related to iTV during our fieldwork can be regarded as small, and we need to work with more participants in our future work, observing and talking with them and others while using different digital technologies has allowed us to start to identify and understand interactions issues which are common across to technologies.

3.2. TV

Attitudes towards watching online on

As there is a growing tendency towards accessing the Internet through traditional TVs, we decided to explore the attitudes of our participants towards interacting with online iTV using their own TVs. At first, they found it difficult to imagine an online TV. The computer was the device for doing online activities. However, after having had some contact with online iTV through computers, all participants showed a big interest in doing the same through their TVs. As stated earlier, they were keen to re-watch their favorite TV programs or watch those they missed. It is worth noting that rather than being afraid of digital technologies, our participants wanted to explore what they could do with them3.

3.3.

Privacy, family and social inclusion

Our participants are worried about their privacy when they go online. Most of the participants do not use Facebook because they do not relish the idea of letting unknown people read their messages or personal information. However, they were interested in showing other participants, their children and grandchildren what they do in gora and share with them (e.g. e-mail) information which can be regarded as personal (e.g. a presentation with photos of their grandchildren). Privacy is important, independent of technology and strongly connected with who should read what.

5. NEXT STEPSWe are gathering more ethnographical data. We expect to combine informal conversations with more structured interviews and focus groups in order to deepen and widen our first-hand observations and in-situ conversations. We are also currently analyzing the diaries filled by our participants. This analysis should help us cover more activities and gather more quantitative data (e.g. frequency of use of TV). We plan to conduct much more activities (e.g. workshops) related to iTV so

3

During a session on Course on Gardens of the World, participants were interesting in knowing how to display presentations (they create with MS Power Point) on the TV to show them when people paid them a visit

62

that we can explore further the use of several iTV services, platforms and technologies. We will also design quantitative studies to understand the effect of observational and conversational data on the interactions of older and younger people on the prototypes we will design.

Life 2.0: Geographical positioning services to support independent living and social interaction of elderly people (CIP ICT PSP20094270965). http://www.life2project.eu/. Last accessed on 1-Feb-2011 Millen, D. 2000. Rapid Ethnography: Time Deepening Strategies for HCI Field Research. In DIS 2000, New York, 280-286. Moggridge, B. 2007. Designing Interactions. Cambridge, MA. The MIT Press. Radio Televisin Espaola. http://www.rtve.es/alacarta/. Last accessed on 27-Feb-11 Rice, M., & Alm, N. 2008. Designing New Interfaces for Digital Interactive Television Usable by Older Adults, Computers in Entertainment (CIE) - Social television and user interaction, Volume 6 Issue 1, January 2008. Righi, V., Maln, G., Ferreira, S., Sayago, & S., Blat, J. 2011. Preliminary findings of an ethnographical research on designing accessible geolocated services with older people. 14th International Conference on Human-Computer Interaction. Orlando, USA. Accepted for publication. Sayago, S., & Blat, J. 2009. About the relevance of accessibility barriers in the everyday interactions of older people with the web. W4A2009 - Technical, (p. 104-113). Madrid, Spain. Sayago, S., & Blat, J. 2010. Telling the story of older people emailing: An ethnographical study. International Journal of Human-Computer Studies, Volume 68, Issues 1-2, January-February, (p. 105 -120). Svensson, M., & Sokoler, T. 2008. Ticket-to-Talk-Television: Designing for the circumstantial nature of everyday social interaction. Proceedings of the 5th Nordic conference on Human-computer interaction: building bridges, October 20-22, Lund, Sweden. Televisi de Catalunya. http://www.tv3.cat/videos. Last accessed on 27-Feb-11.

6. ACKNOWLEDGMENTSThis work has received the support from the Ministry of Foreign Affairs and Cooperation and the Spanish Agency for International Development Cooperation (MAEC-AECID), and the Commission for Universities and Research of the Ministry of Innovation, Universities and Enterprise of the Autonomous Government of Catalonia. We are indebted to our participants and gora for their participation in our research, and our colleagues at the Interactive Technologies Group for their support and feedback. We also thank the reviewers of this paper for their comments and suggestions.

7. REFERENCESArias, J. 2006. Diseo y evaluacin de la interfaz de usuario de un buscador de vdeos sencillo para personas mayores. MsC degree project. Universitat Pompeu Fabra. Barcelona Televisi. http://www.btv.cat/alacarta/. Last accessed on 27-Feb-11 British Broadcasting Corporation (BBC) Television. http://www.bbc.co.uk/iplayer/tv. Last accessed on 18-Apr2011 Charmaz,K., Mitchell, R.G., 2007. Grounded theory in ethnography. In: Atkinson, P., Coffey, A., Delamont, S., Lofland, J., Lofland, L.(Eds.), Handbook of Ethnography. SAGE Publications, London, 160175 Dickinson, A., Newell, A. F., Smith, M. J., & Hill, R. L. 2005. Introducing the Internet to the over-60s: Developing an email system for older novice computer users. Interacting with Computers, 6(17), p. 621-642 Kurniawan, S. 2007. Older Women and Digital TV: A Case Study. ASSETS07, (251-252). Tempe, Arizona.

63

Multipleye Concurrent Information Delivery on Public DisplaysMorin OstkampMnster University of Applied Sciences Stegerwaldstr. 39 48565 Steinfurt, Germany

Gernot BauerMnster University of Applied Sciences Stegerwaldstr. 39 48565 Steinfurt, Germany

morin.ostkamp@fh-muenster.de ABSTRACTThe visible area of a monitor is often called the displays visual real estate. On many contemporary desktop systems, this visual real estate is subdivided into small units, each showing dierent types of information independently (e.g. time, user name, network status). Though there may be many independent units on display in parallel, each of them is most commonly used to transport only one information at a time and is dedicated for one particular user the visual real estate occupied by a web browser cannot be used concurrently by a word processor. This imposes a constraint on the amount of information delivered by the medium: A public display can satisfy peoples curiosity only one by one, but not simultaneously without resizing the used visual real estate. Thus, people are compelled to wait in front of such displays until the information they individually desire is shown. This loss of time is often an annoyance to the viewers, since they could have spent this dwell-time on other more meaningful things. In a project called Multipleye we try to discover ways of multiplexing information visually, e.g. by frequency-, space-, time-, and code-division multiplexing. According to the viewers choice, a mobile app demultiplexes the individual information from the multiplexed image. The increased amount of information transmitted per time can reduce unnecessary dwell-time in front of public displays. This paper presents a demonstrator based on frequency-division multiplexing, discusses rst results and proposes further work.

gernot.bauer@fh-muenster.de 1. INTRODUCTION & RELATED WORKThere is a substantial body of research on the usability of public displays today. Some of the work focuses on how the visual representation of information can be optimized when perceived by more than one person at a time. In [5] Izadi et al. present Dynamo, a system allowing two ore more persons to work on a particular task by using a communal interactive interface. However, this approach only focuses on explicit user interaction with dedicated, spatially divided screen areas. Kray et al. investigate peoples preferences for logically subdivided displays with the Hermes and GAUDI system in [6]. Their approach is however not intended to work with more than two information at a time. A similar strategy is used by Linden et al. for the UBI-hotspot system as presented in [8]. This outdoor system can be used by multiple passers-by while each of them explicitly interacts with a dedicated display area. Thus, the size of the useable display area shrinks with each new user. Peltonen et al. describe CityWall [10], which also allows passers-by to interact with an outside display. But their approach does not provide any means to protect the visual real estate of one user from others. Vogel and Balakrishnan introduce the Interactive Public Ambient Display in [12]. Their system allows users to access information on public displays while the employed visual real estate can still be used by other viewers because of partial transparency. However, this way the number of parallel information is quite limited and the resulting image may become cluttered and confusing since different contents may overlap. In [9] Oliver et al. demonstrate how dierent devices can be orchestrated to implement a crossmodal public-private display. Their proposed CrossFlow system gives navigational directions to multiple users by using time-division multiplexing and a so called crossmodal cue (e.g. the vibration of the users mobile phone). Nevertheless, this multiplexing approach does not really deliver multiple information in parallel. Instead, it presents each information in separate time slots of 0.8 seconds. Due to their interaction concept, all of the above mentioned results can be applied to Computer Supported Collaborative Work (CSCW) environments. However, there is little work on shared use of public displays without direct user interaction. An approach without direct user interaction could be of interest for public spaces such as government agencies, waiting areas, or airports. At such locations, public displays usually show a linear playout of predened content. In most cases, the viewer cannot inuence the presentation, but only perceive it. If the information appears to be irrelevant to the viewer, she is likely to divert her attention

Categories and Subject DescriptorsH.5.2 [Information Interfaces and Presentation]: User Interfaces

General TermsDesign

Keywordspublic display, multiplexing, dwell-time, channel capacity

from the screen. The viewer as well as the display operator strive to avoid this irrelevance, for it means a waste of the viewers time and a waste of the operators resources. An improvement would be to allow the viewer to switch contents as easily as to switch TV channels. However, due to varying interests, there may be some potential for conicts when picking the TV channel. To avoid such a clash of interests in public spaces, this calls for special means of content selection and distribution. Multipleye is exactly about this: a system capable to multiplex and demultiplex information visually, thus increasing the amount of information delivered simultaneously (see Figure 1). In telephone engineering multiplexing is used to transmit scores of phone calls through one wire at the same time. In a similar way, we try to transport more than one information at once using the same public display. The projects name is a neologism of multiplexer and eye, the latter one alluding to our visual approach to multiplexing.

Multiplexed Image

Demultiplexed Images

Figure 1: The basic idea of Multipleye.

image, and faultlessly would state, that the demultiplexed information can be perceived with accuracy, completeness, and consistency as dened by Wand and Wang in [13]. In many technical domains a common method to transmit multiple information over the same medium is to use multiplexing. A couple of strategies can be employed: frequencydivision (FDM), space-division (SDM), time-division (TDM), code-division (CDM), and polarization (see [1, 2, 14]). As a start, the Multipleye project focuses on frequency-division multiplexing, the results being discussed in the remainder of this paper.

2.

IDEA

Our aim is to allow more than one viewer at a time to use the public displays visual real estate for information retrieval. Thus reduce the dwell-time a viewer has to spend waiting until her individually desired information is shown on the public display. The approach pursued in this paper is to increase the information amount per time on public displays. Beforehand, it is necessary to dene how this amount could be measured and named. Possible well-known terms are bit, bandwidth, or channel capacity. Their eligibility is discussed i