• Published on

  • View

  • Download

Embed Size (px)


  • Slide 1

EXPERIENCE SAMPLING VASSILIS JAVED KHAN PANOS MARKOPOULOS THE METHOD AND ITS TOOLS Slide 2 INDUSTRIAL DESIGN DEPARTMENT 1 290609 INDEX ESM: definition, uses & tools Why is there a need for another ES tool? ESRM & Reconexp Our plans for developing a better ES tool Slide 3 INDUSTRIAL DESIGN DEPARTMENT 2 290609 ESM HISTORY Early 1970s pagers were introduced University of Chicago wanted to study flow experiences What were the most enjoyable moments of participants day Slide 4 INDUSTRIAL DESIGN DEPARTMENT 3 290609 ESM DEFINITION A method to solicit respondents answers at random moments of the day [Hektner J.M., Schmidt J.A., Csikszentmihalyi M.,, Experience Sampling Method, Measuring the quality of everyday life, 2007, SAGE, ISBN 1412925576] ESM is a survey method, i.e. a way to collect self-reported data from participants that is executed over sustained periods of time Computerized ESM Slide 5 INDUSTRIAL DESIGN DEPARTMENT 4 290609 WHERE/HOW ESM IS USED? Psychology & Social psychology Feelings of people Activities and what makes them happy Medical Assessing the burden of different illnesses The health consequences of stress Political For evaluating policies HCI, UX & design Assessing the experience of a service or a product Eliciting requirements for a service or a product Slide 6 INDUSTRIAL DESIGN DEPARTMENT 5 290609 ON MY WAY TO LEIDEN Slide 7 INDUSTRIAL DESIGN DEPARTMENT 6 290609 ON MY WAY TO LEIDEN How accurate? How easy? Slide 8 INDUSTRIAL DESIGN DEPARTMENT 7 290609 Slide 9 INDUSTRIAL DESIGN DEPARTMENT 8 290609 Slide 10 INDUSTRIAL DESIGN DEPARTMENT 9 290609 Slide 11 INDUSTRIAL DESIGN DEPARTMENT 10 290609 DRAWBACKS & BENEFITS OF ESM Self-selection bias Nonresponse and selective nonresponse Obtrusive High cost of implementation +Extremely rich data over time (subjective +objective data) Slide 12 INDUSTRIAL DESIGN DEPARTMENT 11 290609 RELATED TOOLS GENERIC Academia MyExperience DPUIS PMAT ESP PsychLab Commercial Entyware by Techneos CUSTOM (for a single study) Slide 13 INDUSTRIAL DESIGN DEPARTMENT 12 290609 MyExperience (beta) FEATURES XML Questionnaire development Availability of several sensors (GPS, device usage information) Supports photo, video and audio capture Plug-in architecture Windows Mobile devices Slide 14 INDUSTRIAL DESIGN DEPARTMENT 13 290609 MyExperience XML example Slide 15 INDUSTRIAL DESIGN DEPARTMENT 14 290609 FUTURE ES TOOL Why another tool? Conditions are right: truly affordable Shortcomings of previous tools New methodological developments Slide 16 INDUSTRIAL DESIGN DEPARTMENT 15 290609 CONDITIONS ARE RIGHT: More people have powerful devices Globally, research house Ovum expects 171.9 million smartphones to ship this year, up 23% on 2008 and about 15% of the worldwide mobile phone market in 2009 [reuters.com] http://www.appleinsider.com/articles/08/02/14/google_iphone_usage_shocks_search_giant.html Slide 17 INDUSTRIAL DESIGN DEPARTMENT 16 290609 CONDITIONS ARE RIGHT: Internet on mobile devices is now affordable Slide 18 INDUSTRIAL DESIGN DEPARTMENT 17 290609 PREVIOUS TOOLS SHORTCOMINGS Difficult to install Too technical to execute Little support Need of expensive devices Slide 19 INDUSTRIAL DESIGN DEPARTMENT 18 290609 NEW METHODOLOGICAL DEVELOPMENTS Providing feedback to participants improves their motivation [Hsieh, G., Li, I., Dey, A., Forlizzi, J., and Hudson, S. E. 2008. Using visualizations to increase compliance in experience sampling. UbiComp '08] Reconstruction of sampling day [Khan, V.J., Markopoulos, P., Eggen, B., de Ruyter, B., IJsselsteijn, W.A. (2008) Reconexp: A way to reduce the data loss of the Experiencing Sampling Method, MobileHCI '08] Defining inconvenient times [Isomursu, M., Thti, M., Vinm, S., and Kuutti, K. 2007. Experimental evaluation of five methods for collecting emotions in field settings with mobile applications. Int. J. Hum.-Comput. Stud. 65, 4 (Apr. 2007), 404-418] Slide 20 INDUSTRIAL DESIGN DEPARTMENT 19 290609 RECONEXP STUDY: OUR EXPERIENCE RECON STRUCTING EXP ERIENCE A first tool supporting the ESRM method Slide 21 INDUSTRIAL DESIGN DEPARTMENT 20 290609 GOAL Investigate communication needs of busy parents (under context) What information are busy parents willing to automatically exchange with each other during a typical day of theirs? Slide 22 INDUSTRIAL DESIGN DEPARTMENT 21 290609 Slide 23 INDUSTRIAL DESIGN DEPARTMENT 22 290609 Slide 24 INDUSTRIAL DESIGN DEPARTMENT 23 290609 Slide 25 INDUSTRIAL DESIGN DEPARTMENT 24 290609 Slide 26 INDUSTRIAL DESIGN DEPARTMENT 25 290609 METHOD EXPERIENCE SAMPLING + DAY RECONSTRUCTION Slide 27 INDUSTRIAL DESIGN DEPARTMENT 26 290609 + DAY RECONSTRUCTION Slide 28 INDUSTRIAL DESIGN DEPARTMENT 27 290609 RESULTS Busy parents are willing to automatically exchange information which will help them to: Infer the availability of partner Feel reassured about the well being of family members Help coordinate tasks in the evening Slide 29 INDUSTRIAL DESIGN DEPARTMENT 28 290609 Frequency of chosen information statements Slide 30 INDUSTRIAL DESIGN DEPARTMENT 29 290609 SITUATIONAL DATA ANALYSIS Slide 31 INDUSTRIAL DESIGN DEPARTMENT 30 290609 SITUATIONAL DATA ANALYSIS Slide 32 INDUSTRIAL DESIGN DEPARTMENT 31 290609 RECONEXP SUMMARY Supports personalization of available answers Supports feedback to participants Supports filling out unanswered queries Slide 33 INDUSTRIAL DESIGN DEPARTMENT 32 290609 WHAT WOULD NEXT ES TOOL LOOK LIKE? Slide 34 INDUSTRIAL DESIGN DEPARTMENT 33 290609 TOOL COMPONENTS Questionnaire Developer Event triggering editor Choice of presentation medium (mobile | pc) Real time feedback to participants Data analysis & Visualization tools Notification manager (participant and researcher) Methodology wizard Experience granularity Participants state of mind-based triggering of queries Emphasis is on mobile studies Slide 35 INDUSTRIAL DESIGN DEPARTMENT 34 290609 INTERESTING? HTTP:// ESRM.ID.TUE.NL p.markopoulos@tue.nl v.j.khan@tue.nl Slide 36 INDUSTRIAL DESIGN DEPARTMENT 35 290609 ESRM Experience Sampling and Reconstruction Method A method combining ESM with DRM Slide 37 INDUSTRIAL DESIGN DEPARTMENT 36 290609 DAY RECONSTRUCTION DRM respondents first revive memories of the previous day by constructing a diary consisting of a sequence of episodes. Then they describe each episode by answering questions about the situation and about the feelings that they experienced, as in experience sampling. [Kahneman, Daniel, Alan B. Krueger, David A. Schkade, Norbert Schwarz and Arthur A. Stone. 2004. A Survey Method for Characterizing Daily Life Experience: The Day Reconstruction Method. Science. 306:5702, pp. 1776780.] Shortcomings: data for one day, requires one-to-one access with participant Slide 38 INDUSTRIAL DESIGN DEPARTMENT 37 290609 PLATFORM? Windows Mobile Blackberry iPhone Palm Nokia Slide 39 INDUSTRIAL DESIGN DEPARTMENT 38 290609 CRITERIA Market share Development tools Cost Form factor Slide 40 INDUSTRIAL DESIGN DEPARTMENT 39 290609