A deep dive into questions by @cjforms at UxLx

  • Published on
    21-Apr-2017

  • View
    2.976

  • Download
    0

Embed Size (px)

Transcript

PowerPoint Presentation

A deep diveinto questionsWorkshop at UxLx 2014 led by Caroline JarrettHow to ask better questions, andhow to assess user experience using surveys

1 CHI 2010 Caroline Jarrett

Introductions(Im Caroline Jarrett - @cjforms)Work with your neighbourYour name and roleA random thing about yourself2

AgendaIntroductionsHow to ask better questionsBreakHow to assess user experience using surveysWrap up3

A survey I saw recently4How do we know its a survey?

5

And it continued.6

Ill hand out an invitation I received recently by emailWork in pairsDecide whether it is a survey or something else7

In this exercise, participants reviewed an email from a journeal that said it was a survey, but seemed to us more like a form. CHI 2010 Caroline Jarrett7

AgendaIntroductionsWhat is a survey?How to ask better questionsThe steps to answer a questionImprove step 1: read and understandImprove step 2: find the answerImprove step 3: judge the answerImprove step 4: place the answerUnderstand why people answer BreakHow to assess user experience using surveysWrap up8

There are four steps to answer a questionStep1. Read and understand2. Find an answer3. Judge the answer4. Place the answer

Adapted from Tourangeau, R., Rips, L. J. and Rasinski, K. A. (2000)The psychology of survey response

There are four steps to answer a questionStepA good question 1. Read and understandis legible and makes sense2. Find an answerasks for answers that we know3. Judge the answerasks for answers were happy to reveal4. Place the answeroffers appropriate spaces for the answers

Adapted from Tourangeau, R., Rips, L. J. and Rasinski, K. A. (2000)The psychology of survey response

Are you ?

11

Lets review a questionThere is a question coming up on the next slideI will ask you to think about ONE of these four stepsRead and understand Find the answerJudge the answerPlace the answerPlease think about any problems in that particular step

12

13

AgendaIntroductionsWhat is a survey?How to ask better questionsThe steps to answer a questionImprove step 1: read and understandImprove step 2: find the answerImprove step 3: judge the answerImprove step 4: place the answerUnderstand why people answer BreakHow to assess user experience using surveysWrap up14

Legibility is also important

15Hermann grid illusion

CHI 2010 Caroline Jarrett15

AgendaIntroductionsWhat is a survey?How to ask better questionsThe steps to answer a questionImprove step 1: read and understandImprove step 2: find the answerImprove step 3: judge the answerImprove step 4: place the answerUnderstand why people answer BreakHow to assess user experience using surveysWrap up16

17

In your last five days at work, what percentage of your work time do you estimate that you spend using publicly-available online services (not including email, instant messaging and search) to do your work using a work computer or other device?

The approximate curve of forgetting

AgendaIntroductionsWhat is a survey?How to ask better questionsThe steps to answer a questionImprove step 1: read and understandImprove step 2: find the answerImprove step 3: judge the answerImprove step 4: place the answerUnderstand why people answer BreakHow to assess user experience using surveysWrap up19

I saw this question on an employee survey

20

Lets create a list 21

AgendaIntroductionsWhat is a survey?How to ask better questionsThe steps to answer a questionImprove step 1: read and understandImprove step 2: find the answerImprove step 3: judge the answerImprove step 4: place the answerUnderstand why people answer BreakHow to assess user experience using surveysWrap up22

Test with users to make sure youoffer the right answer options23

24

25

Offer the right widget to collect the answerKnowledge of what users want to tell youHow many answers?OfferWe know all the answers that users are likely to give usThey only have one answerRadio buttonsThey may have more than oneCheck boxesWere not sureText boxes

26Allen Miller, S. J. and Jarrett, C. (2001) Should I use a drop-down?http://www.formsthatwork.com/files/Articles/dropdown.pdf

27

Grids are often full of problems at all four steps28

Grids are a major cause of survey drop-out29Source: Database of 3 million+ web surveys conducted by Lightspeed Research/KantarQuoted in Coombe, R., Jarrett, C. and Johnson, A. (2010) Usability testing of market research surveys ESRA Lausanne

But its the topic that matters most30Source: Database of 3 million+ web surveys conducted by Lightspeed Research/KantarQuoted in Coombe, R., Jarrett, C. and Johnson, A. (2010) Usability testing of market research surveys ESRA Lausanne

AgendaIntroductionsWhat is a survey?How to ask better questionsThe steps to answer a questionImprove step 1: read and understandImprove step 2: find the answerImprove step 3: judge the answerImprove step 4: place the answerUnderstand why people answer BreakHow to assess user experience using surveysWrap up31

Response relies on effort, reward, and trust32

TrustPerceivedeffortPerceivedrewardDiagram from Jarrett, C, and Gaffney, G (2008) Forms that work: Designing web forms for usability inspired by Dillman, D.A. (2000) Internet, Mail and Mixed Mode Surveys: The Tailored Design Method

An interesting subject helps in all the areas33

Shared interestsinspire trustInterestingtopicstakeless effortInteresting subject =intrinsicallyrewarding

What about response here?34

35Your answers to this survey are important for our work

But whats in it for me? And Im really ready for a coffee.

AgendaIntroductionsWhat is a survey?How to ask better questionsBreakHow to assess user experience using surveysWrap up36

Lets start with a standard option: SUSThe System Usability Scale (SUS) was created in 1986It has been shown to be valid and reliableYou get a score between 0 and 100You can compare your SUS score with other systems37Brooke, J. (1996). SUS: A "quick and dirty" usability scale. In Usability Evaluation in Industry. P. W. Jordan, B. Thomas, B. A. Weerdmeester and A. L. McClelland. London:, Taylor and Francis.

Jeff Sauro (@measuringux) has done a lot of work with SUS

38http://www.measuringusability.com/sus.php.Jeff provides tools for scoring SUSHe has adapted it to websitesSUS scores are not percentages

There are other commercial products with wider concepts of UX

39SUPR-Q includes credibility and loyaltylicenced producthttp://www.suprq.comWAMMIonline serviceincludes access to standard databases (extra for SUPR-Q)http://www.wammi.com

Your task: find an explanation of the difference between the European Commission and the European UnionUse this site: http://ec.europa.euDecide whether you had a good or bad experience

40

I will ask you to fill in SUS (original) or SUS (Sauro)

41

My journey into user experience started a long time ago, with usabilityThe extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use(ISO 9241:11 1998)42

Introduction to usability testing(c) Effortmark Limited 2004

Working mostly in government, we were interested in effectiveness and efficiency43The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use(ISO 9241:11 1998)

But what about user experience?What about satisfaction?44The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use(ISO 9241:11 1998)

Picture credit: Flickr jek in the box

45

Satisfaction is a complex matterCompared experience to what?Resulting thoughts(nothing)IndifferenceExpectationsBetter / worse / different NeedsMet / not met / mixtureExcellence (the ideal product)Good / poor quality (or good enough)FairnessTreated equitably / inequitablyEvents that might have beenVindication / regret

46Adapted from Oliver, R. L. (1996) and (2009)Satisfaction: A Behavioral Perspective on the Consumer

Example: bronze medal winners tend to be happier than silver medal winners

47Nathan Twaddle, Olympic Bronze Medal Winner in BeijingPhoto credit: peter.cipollone, Flickr Matsumoto D, & Willingham B (2006). The thrill of victory and the agony of defeat: spontaneous expressions of medal winners of the 2004 Athens Olympic Games.

The first question was about rating satisfactionWhat were they asking us to rate?Just a guess from what you recall

48

49

The challenge of UX and surveys: which bit to measure?The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use(ISO 9241:11 1998)???

Some ideas about what we could measureIn the definitionGoDaddy customer supportGoDaddy as a provider of domain namesProductThis contact with help deskOverall experience of moving a domain to GoDaddyUsersWhat proportion of customers contact supportDemographics (example: type of job)GoalsReason for contacting helpReason for looking at GoDaddyEffectivenessWhether support fixed the problemWhether GoDaddy offers the right productsEfficiencyWhether it took a reasonable timeWhether the product is priced correctlySatisfactionHelpfulness of support personLikely to purchase again / recommendContext of useHome/office; alone/helpedBusiness / personal

51

52In the definitionInformation to help the European Commission design a better websiteProductUsersGoalsEffectivenessEfficiencySatisfactionContext of use

Write questions for each topicThen get another team to try your questionnaire53

Find out about users goalsTip54

Ask about recent vivid experienceTip55

Image credit: Fraser Smith

Interview firstTip56

Test everythingTip57

Caroline JarrettTwitter @cjformshttp://www.slideshare.net/cjformscarolinej@effortmark.co.uk

58

58 CHI 2010 Caroline Jarrett

More resources on http://www.slideshare.net/cjforms59

59 CHI 2010 Caroline Jarrett