Housekeeping • Seminar Topics doodle poll on facebook page – sign up by tomorrow night (Fri) please, and I’ll make the schedule this weekend

• Option to propose a novel topic. One slot on the last day – I may free up another couple on other days depending on interest levels

• Post-mortem of 1st MP1 Protocol testing• Preparation• Volunteers• Those rating sheets• Critiques

Agenda• Today:• Finish off contextual interviews• Diary Studies• Discussion of diary studies reading (assigned for Tues)• Questionnaire handout (start in class – complete for Tuesday)

• Next Tuesday:• Finish off Questionnaires, Methodology Matters

Contextual inquiry

Interviewees are interviewed in their context, when doing their tasks, with as little interference from the interviewer as possible.

Allows probing of “why?” Can be real-time or record

interesting actions for later discussion

“Typical” 4 phased approach Traditional interview

Get an overview, establish trust, start recording Switch to a master-apprentice relationship

Tell them what you want to observe Make sure to establish when ok to interrupt Observe, ask questions

Take notes Balance need to understand with impact of

interruptions Summarization

Go over observations and your understanding with participant

Make sure that you go it right

Other ways of providing context If natural observation not possible,

can ask them to demonstrate specific tasks of interest

Can provide task scenarios and ask them to perform

“Think aloud” aloud protocols

Other ways of getting observational data

Logging Screen recording (check out

Camtasia) Trace data

CSCI 4163/6610, Winter 2014

Advertising Diary

Did you remember to fill it out? When did you do it?

As encountered? In a batch?

Were the instructions clear? What was easy/hard about it? Do you think it captured your

exposure to advertising?

Diary studies…

Participants collect data about events As they happen In the context of the event (in situ)

Can think of like a small, longitudinal questionnaire

Often used to prompt interview discussion (similar to observations in a contextual inquiry)

Can help understand rare/infrequent events

Data collection methods

Survey style forms Paper (little training required, but hard to

monitor) Online entry forms Google docs

Photographs, video Digital/disposable camera, mobile phone

Audio recording Voice recorder, mobile phone

Aggregating data online Blogs (text, audio, video) , Twitter Rich and timely information

Type of data recorded

When Date/time Duration Activity/task

What Activity/task feelings/mood Context (environment/setting)

When is data recorded?

Randomly In response to prompts

At specific intervals Based on activity

General types of diary studies Unstructured

Ask participants to report on everyday activities Trying to elicit general themes

Structured Ask participants to report on everyday activities

by answering specific questions about the activity

Combination of question types Can also be used as a form of usability

tests and problem reports Ask them to complete a task and report results,

identify bugs, etc.

Variations based on purposeFeedback (now) Elicitation (later)

Record everything in situ

No follow up Focus on the “what”

data Mostly structured Can be burdensome

on participants and researchers

Record aspects or triggers of an activity

Follow up with an interview to gain more information

Data captured is used to elicit the “why” and the how”

Mostly unstructured Can be problems with

participant recall

Cultural Probe Diary study++ Used to not only record

activities, but capture more of “felt life” Stimulate thought as well as capture

experiences http://www.hcibook.com/e3/casestudy/cultural-probes/ - pack contained

a small disposable camera and a listening glass that participants used to listen at walls and doors and write what they heard. It also contained a small solid state recorder packaged in a sleeve that said 'dream recorder'.

Was used by designers at the Royal College of Art, London, to study the way people see their own homes. The results were used to enable designers to get a 'feel' of the meaning of home for many people.

Experience sampling method (measure feelings, moods)

Technology Probes (prototypes, experimental artifacts)

Coordination Challenges

Keeping participants on track Periodic reminders Feedback about the level of detail in responses Progressive incentives, surprise

incentives/gifts (may or may not be allowable by BREB)

Adapting to changes Start analysis as soon as first results arrive May need to re-evaluate the diary format if the

data being captured is not what you expected!

Advantages

Time efficient for researchers Lower cost than direct observations

Can have a broad geographic distribution

Supports contextual reports over time Natural environment

Can give rich data about contexts of use

Disadvantages Participant fatigue

Missed responses (frequency data is generally unreliable – lower bound)

Study drop outs “after the fact” reporting to catch up

Behaviour adjustment Participants need reminders Can be expensive for long durations Volume of collected data can be

overwhelming

Study protocol

Introductory letter Consent form Needs to motivate their participation

Very specific instructions Can be helpful to give an example of a

filled out form (take care not to prime/limit)

Diary form / collection aids

Forms of analysis

Track temporal patterns Look for semantic patterns in visual

data Combine with interview analysis

Diary reading

When Participants Do the Capturing: The Role of Media in Diary Studies What was their motivation? What were their research questions? What was their approach? What were their findings? What were the strengths of their study

design? What were the weaknesses? Do you trust their findings?

Question writing exercise 1. Answer the questionnaire (these are

BAD questions!) 2. Take a few minutes to think about what

made the questions bad/hard to answer/hard to analyze

3. look at 6-17 on the questionnaire design checklist For each checklist item, identify which

questions are examples of poor design 4. On Tuesday, we will discuss this as a

class