Diary studies

Rikard HarrNovember 2010

© Rikard Harr 2010 3

Outline

• The Diary study: benefits, challenges and alternatives

• The papers: aims and use of diary study– Short summaries

• Group task

© Rikard Harr 2010 4

The Diary study

Benefits• An approach for studying people on the move and in

situ (all)• A method for understanding participant behavior that

minimizes the effect of observers (Carter and Mankoff, 2005)

• Demands less resources than alternative methods (Carter and Mankoff 2005)

• Helps create empirical evidence• Helps develop trust, a suitable first method (Hess and

Wulf, 2009)

© Rikard Harr 2010 5

Alternatives for understanding everyday human behavior• Observations• Ethnographic approaches, video recorded• Contextual inquiries• Survey studies• Experience Sampling Method (ESM)• Probes

© Rikard Harr 2010 6

Challenges

• How to find a balance between feedback studies and media elicitation studies– People answer/record questions/things as things happen– People record events and are interviewed afterwards

• How to gather user entries in feedback studies– Dial to a voice mail system– Create audio recordings– Creating written entries

• Which media to use in media elicitation studies– Audio– Photo– Tangible objects

© Rikard Harr 2010 7

The papers and Diary studies

• Czerwinski et al. (2004) make use of it for characterizing how people interleave multiple tasks amidst interruptions

• Sohn et al. (2008) make use of it for understanding mobile information needs and how they are addressed

• Hess and Wulf (2009) structures it and add camera and pen (?)

• Carter and Mankoff (2005) explores the effect of using different media to capture events and how this affect the diary study method

© Rikard Harr 2010 8

Applying the method: Czerwinski

• Aim, to characterize how people interleave multiple tasks amidst interruptions

• Baseline survey• Used diaries during a week• Also captured users descriptions of their work• 11 participants were used (3 female)• All reported on participation in at least 3 projects• Age from 25 to 50• A spreadsheet for each day was created with

columns: Time of task start, difficulty switching to the task, what documents were included in the task, what was forgotten if anything, comments, and number of interruptions experienced

• A code scheme had been developed• Two experimenters coded the first day diaries

© Rikard Harr 2010 9

Applying the method: Hess and Wulf• Aim, to evaluate a structured media diary for

capturing participants’ social video consumption• Date, Context, Event, Persons, Comment, Picture,

Sticker• Recruited participants via an article in the

newspaper• 9 households, 17 participants• Short introduction before handing out ”the boxes”• Age, between 10 and 45• Range of study, 2-4 weeks• Short semi-structured interview (10 minutes)• Diaries were accepted very well• 17 participants made 189 diary entries• Strong differences in the level of detail in diaries• Stickers were used in 15 cases• Got photos from 6 out of 9 (1-6)

© Rikard Harr 2010 10

Applying the method: Carter and Mankoff• Aim: seek to understand how context information and

episodic memory prompts captured by participants vary with media, in what way different media supports recall and how different media affect the diary study process

• Wants to improve the technique, based on three studies– Observed and interviewed researchers in two studies– Conducted their own study

Results• Audio elicitation suffers from recognition problems, but

encourage more discrete event capturing• Tangible objects unravels users attitudes and beliefs

better, but are not good for remembering events• Photos are easiest to capture and recognize• For studies in which detail is important audio/photo is

preferable

© Rikard Harr 2010 11

Applying the method: Carter and Mankoff• Location information does not add much for

recalling an episode• People are more keen on providing feedback

answers if there are fewer events• To support recall it is good to tag a captured

event with a brief annotation• Researchers should be able to review captured

data as well as annotations before the interview• Automatic time-stamping is important in all cases

© Rikard Harr 2010 12

Group task• You want to study the disruptive effects of various forms

of interruptions for information workers. This in order to provide implications for design of future ICT

• You have no budget but some time over• You think that factors such as relation, location, situation

matter• How would you designs such a diary study and why?

Motivate your decisions and prepare to present it• If you absolutely want to apply another approach or

method, do so and prepare to motivate why• If necessary make own assumptions about the object of

study

© Rikard Harr 2010 13

Applying the method: Sohn et al.

• Aim: To understand mobile information needs and how they are addressed

• Two-week diary study• 20 participants (10 – 10), aged 19-58• Recruited through an online mail list and flyers• Focused on type of information needs, methods

to adress those needs and contextual factors that prompted needs

• Kept diaries when mobile• Conducted introductory, mid-study and final

interview with all

© Rikard Harr 2010 14

Applying the method: Sohn et al.

• Provided several examples of what to record• Used a text messaging scheme• Messages were sent to an email address which

posted them on a website• Text messages should be composed to support

answering a set of questions at the end of the day• Researchers sent five reminding messages every

day• At the end of the day participants logged in and

answered six questions about their entries• Applied a reward system• 421 diary entries• In interviews clarifications of unclear entries were

made