Usability Testing and QA 10-3-14

Embed Size (px)

DESCRIPTION

Slides from my course "Usability Testing and QA" taught at BAVC 10/3/2014

Text of Usability Testing and QA 10-3-14

  • 1. UsabilityTesting andQA Apracticalapproach towardscreatingbetteruserinterfaces

2. WhoamI? ShilpaThanawala @skthana 3. Andyou? Name Roleatyourcompanyoragency Background(eg.designer?developer?etc.) Experiencewithusabilitytesting Howyouthinkyoullusewhatyoulearntoday Whythisclass? 4. Housekeeping Initialandsigntheroster Evaluationsattheendoftheday Cellphonesonsilent,callsoutside EveryoneknowswherethingsareatBAVC? LoadthesharedGoogleDoc (forquestions,backchannelinteractions,sharinglinks) http://bit.ly/uxoct03 5. UsabilityTesting Somethoughtstostartusoff... 6. Designersknowtoomuchabouttheirproducttobeobjectivejudges; thefeaturestheyhavecometoloveandprefermaynotbeunderstood orpreferredbyfuturecustomers. DonNorman,nngroup.com Imagecredit:templatemonster.com 7. Thetestofthemachineisthesatisfactionitgivesyou.Thereisntany othertest.Ifthemachineproducestranquilityitsright.Ifitdisturbs youitswronguntileitherthemachineoryourmindischanged. RobertM.Pirsig,ZenandtheArtofMotorcycleMaintenance:AnInquiryIntoValues Imagecredit:Optimum7.com 8. Supposingisgood,butfindingoutisbetter. MarkTwain Imagecredit:Mediamatic 9. UsabilityTesting Somehistoricalhighlights... 10. Early20thCentury Improvementsinindustrialefficiency InWWI,reducedworkmotionsforsoldierstoassemble& takeapartweaponsinthedark. http://www.measuringusability.com/blog/usabilityhistory.php 11. 1947 JohnKarlinhowshortphonecordsshouldbe 12. ItisnotsomuchthatMr.Karlintrained midcenturyAmericanshowtousethetelephone. Itis,rather,thatbystudyingthepsychological capabilitiesandlimitationsofordinarypeople,he trainedthetelephone... ExcerptfromNewYorkTimesarticle 13. 1980s Newmethodologiesdevelopedandpublished Usabilitybecomesaprofession Definedasafunctionofefficiency,effectiveness,and satisfaction firstpublishedPsychologyofEverydayThings 1990s Usabilitymatures Diminishingreturnsfromtestingmorethan35participants BooksbyJakobNielsen 14. 2000today SteveKrugsDontMakeMeThink Remotetesting Statisticalanalysisofusabilitydata Mobiledevices,touchandgestures http://www.measuringusability.com/blog/usabilityhistory.php 15. Why? 16. Whyspendvaluabletimeandmoneydoing usabilitytests? Itsthebestwaytolearnhowusersactuallyinteractwith yourwebsite. Watching>interviewing Youdontreallyknowyourusersneeds.Usersarenotallthe same.AndtheyrenotyourMom. Allwebsiteshaveproblems,andtheseriousonesareeasyto find. 17. Whyspendvaluabletimeandmoneydoing usabilitytests? Youlllearnthingsyoudidntknowthatyoudidntknow. Watchingusersgetseveryoneonthesamepage. Watchinguserswillmakeyouabetterdesigner/developer/ productmanager/contentwriter/etc. Usabilitytestingdoesnthavetobeexpensiveortime consuming. 18. WhatTestingCantTellYou 19. LimitationsofUsabilityTesting Wontfixyoursitesproblems(itsnotausertrainingsession) Resultsaredependentonthequestionsyouask(tasks) Cantmodelsocialinteractionsverywell Isntalwaysrepresentativeofrealsituations FormoreonissuestestingthesocialwebseeDanaChisnellstalk 20. When? 21. Whenshouldusabilitytestingbe done? Beforeyoudesignorbuildanything Beforeyouimplementachange Throughouttheproject Afterimplementingafix Throughoutthelifeofthewebsite Assoonaspossible&repeatoften! 22. Demo:Ashortusabilitytest 23. Qualitativevs.Quantitative 24. Qualitativevs.Quantitative QuantitativeUsabilityStudies Designedandconductedtoproducedataforstatisticalanalysis Eachmeasurementrepresentsacountoranamount Distances,weights,quantities,etc. QualitativeUsabilityStudies Focusedonverbaldescriptionsofusersexperiences Eachmeasurementisadescriptionorcategory Words,sentences,feelings,yes/no Categoriesarenotquantitative,eveniftheyareassigned numbers userfocus.co.uk 25. ...qualitativemethodsaremuchbettersuitedforanswering questionsaboutwhyorhowtofixaproblem,whereasquantitative methodsdoamuchbetterjobansweringhowmanyandhowmuch typesofquestions. MoreonQuantitativemethodsat WhentoUseWhichUserExperienceResearchMethods,nngroup.com measuringusability.com 26. UsabilityMetrics 27. Effectiveness,Efficiency,and Satisfaction Effectiveness Accuracy,correctness.Areusersabletoachievetheirgoals? Efficiency Easeofuse,speed.Howmucheffort(time)doesittaketo completeatask? Satisfaction Usersperception.Whatdoestheuserthinkabouttheir experience? Moreat ,usability.gov wikipedia.org/wiki/Usability 28. SomeMetricsforCommonUserTasks FromMeasuringtheUserExperienceTullis&Albert 29. PerformancebasedMetrics TaskSuccess Commonlyused,pass/fail,ordegreesofsuccess,should haveclearendstate. TaskTime Particularlyimportantforrepetitivetasks Errors Incorrectactionsleadingtosignificanttimeloss,additional costs,ortaskfailure Efficiency Effortandtimeneededtocompletethetask Learnability Effortandtimeneededfortheusertolearnhowtousethe interface 30. IssueMetrics Usabilityfindingsbasedonusersbehavior Consistent/repeatable,oridiosyncratic? Remainopenmindedtotheunexpected 31. OtherMetrics SelfReporting:Usersperceptions,feelings Behavioral/Psychological:Eyetracking,stress, unpromptedverbalexpressions Combinations/Comparisons:Interpretationsbasedon morethanonemetric CardSorting:Howtoorganizeinformationinawaythat makessensetousers. A/BTesting:Comparisonoftwoalternatedesigns Accessibility:Usabilityforthosewithdisabilities( , ) CCVAof 2010 WCAG 32. Bias Everystudycontainssomelevelofbias. Participantbackground,knowledge,comfortlevel Taskdefinitions Methodology(testingprocess,sessionlength,howmuch talking) Artifactsoftheprototypeorproduct Environment(lighting,noise,cameras,distractions) TheFacilitator(experience,skill,style) Expectations MeasuringtheUserExperienceTullis&Albert 33. NumberofTestParticipants 34. Whatstherightnumberofusers torecruitforastudy? 35?46?10?20?100? Thosewhofavorasmallnumberfeel80%ofthemost importantusabilityproblemsaredetectedbythefirstfew users. Casesinwhichmoreusersmayberequired: Youneedtobesureyouvecapturedasmanyproblemsas possible Youhavemorethanoneusergroup(persona) Youhavealargenumberofscreens,complexdesign,wide varietyoftasks Youdliketoaccountforevaluatorbias 35. Whywellfocusonqualitativetestingtoday Anyonecandoit Noknowledgeofstatisticsordataanalysisneeded;basedon verbalfeedback,observation,andempathy Cheap,fast,andeasy Greatforsmallbudgets,busyteams,minimalresources... andfrequentrepeats Itgetsthejobdone Morethansufficienttoidentifyawebsitesmostserious usabilityproblems 36. Break 10minutes (backat11:48) http://bit.ly/uxoct03 37. DIYUsabilityTests BasedlargelyonRocketSurgeryMadeEasySteveKrug 38. EstablishyourGoals Whatdoyouwanttolearnaboutyoursite? Canusersfindproductseasily? Isitcleartouserswhatthesiteisabout? Istheinformationorganizedintuitively? Isthisnewfeaturedistracting? Keepanopenmindtoallowforunexpectedresults. 39. DefineyourTasks TasksActivities First,createoneormorescenariosbasedon essential,realworldusergoals. Second,defineactivitiesthatfiteachscenario. Activitiesshouldberealistic Activitiesshouldbeactionable Phrasethemsoasnottogiveawaytheanswer Printeachactivityonitsownpieceofpaper. (Avoidlabelsornumbers.) TurnUserGoalsintoTaskScenariosforUsabilityTestingnngroup.com 40. RecruitingTestParticipants Whatkindsofparticipants? Wheretofindthem? Howmany? Howtocompensatethem? 41. Whatkindsofparticipants? Howrepresentativeofactualusers? MostseriousUIproblemswillbediscoveredbynon representativeusers. Testingwithnonrepresentativeusersisbetterthan obsessingoverfindingrepresentativeusers. Keyrequirements:notpartoftheteam,fluentintheUIs language 42. Wheretofindparticipants? Wheredoyourusershangout? Advertiseonthesite,messageboards,socialnetworks Craigslist 43. Howmanyusers? FormostofusdoingDIYtesting,35participantsisfine. Noshowsorlastminutecancellationsarecommon. Have12backupparticipantslinedup. Phonescreen,scheduling,daybeforereminders,directions, parking,NDAs...useachecklist 44. Compensation Showsappreciationforparticipantstimeandeffort Encouragesparticipantstobeengagedandenthusiastic. Giftcards,freeproductorservice. 45. Equipment&Setup Hint:youdontneedalab Bareminimum(ifyouretheonlyone):acomputerinaquiet room,apenandnotepad,and2chairs Forateam,addaremoteobservationroom,agoodmic,and screensharing(SkypeorGHangout).Andprovide irresistablesnacks. Canusescreencapturesoftware(butmostofthetimeit wontbewatched) Canrecordtheparticipant(butitsnotreallyneeded) 46. MakeitaTeamEffort Trytogeteveryonetoattend. Assignatrustworthyteammembertomanagethe observationroom 47. TheSessionScript Startwithatemplate&edittosuityourneeds Welcome/introduction Logistics,releaseforms Necessarybackgroundinformation Test:generalquestions Test:scenarioactivities Wrapup,followupquestions Concludeandprovidehonorarium 48. SomeTestSessionScriptTemplates Freetemplatesat: , ,infodesign.com SteveKrugssite Booksite guidebyRubin,Chisnell&Spool 49. Debrief Onlyforthosewhoattendedatleastonesession Scheduleitonthesameday(nextdaylatest),6090min. Providefood! Getallobserverstostate(orstick)UXproblemsthey observed Getgeneralagreementonwhicharemostserious&willbe tackledfirst(23maximum) Remindeveryonewhenthenextroundoftestingwilltake place 50. Runyourownusabilitytest 1. Selectawebsitetotest 2. Determinegoals 3. Identifyatleast3activitiescriticaltosuccess 4. Formulatescenariosfortestingeachactivity 51. Lunch 1hour (backat2:10pm) 52. Runyourownusabilitytest 1. Prepareyourscript 2. Rehearseandrefine 3. Printscenario(s)andactivitieson1sheet(wellspacedsoyou cancutthemapart) 53. Runyourownusabilitytest 1. Testtwoparticipants(~20mineach) 2. Takenotes 54. Runyourownusabilitytest 1. Debriefwithinyouroriginalgroup 2. Identify23usabilityissues 55. Tools Testingsessionchecklists , , Usabilitytestingsoftware , , Online/Remoteservices , , , Usability.gov Krugschecklist checklistonbooksiteby Rubin,Chisnell&Spool Silverback(Mac) Morae(Win) Camtasia Usertesting.com Loop11 usabilityhub.com manymore 56. Sohowdowefixit? 57. Fixingusabilityissues Thebadnews:mostissuesnevergetfixed Makethesmallestchangepossibletostopthebleeding Largechangesmightintr