Usability Testing Materials

Usability Testing

Wednesday, February 29, 12

Usability is mostly common sense.


not



not



But you still need a plan.


Test Materials


• Guidelines for observers

• Orientation script

• Background questionnaire

• Data collection instruments

• Agreement & consent forms

• Pre-test questionnaire

• Task scenarios

• Post-test questionnaire

• Debriefing guideWednesday, February 29, 12

Guidelines for observers


what to do not to do


• Introduce yourself

• Find out what they need? Anxious?

• Let them know what they’re in for.

• protocols

• testing methods

• testing reasons


Orientation Script


Guidelines (p. 156-57)


Guidelines (p. 156-57)

• Tone

• Length

• Verbatim


Demonstrate neutrality.



you are



but you do need to care about them.




The act of observing people can’t help but affect the results.


Background Questionnaire


classifiers from screening process


why?



data collection


What data will address the problem statement/

research question?


How will you collect the data?


Who is going to do what?


Performance Data


“objective measures of behavior, such as error rates, time, and counts of observed behavior elements”


Preference data


“more subjective data that measures the participant’s feelings or opinions of the product.”


Performance Data:


• number of errors

• % of tasks completed successfully

• # and type of hints/prompts needed

• # of omitted steps or procedures

• Scores on comprehension tests


• complete a task

• recover from an error

• achieve a criterion level of competence

• spent reading v working

Time to...


Preference Data


• Preference between versions

• Suggestions for improvement

• Number of negative references to product

• Rationales for performance

• Ratings or rankings of product


(see pgs 166 - 167)


What type of information will you collect?



Listen to the client.Pay attention to your site.


First things first:


Where are you going to test?


These documents are used in varying contexts.





Ideally, testing is done in the context the document will be used in.


How are you going to record data?



Morae


Manual data collection


Observe and take notes on every action.


Develop a code.


• You might write “T” when the subject stops reading and does not perform any action to signify that they were thinking

• You might write “F” when the subject flips back and forth to locate something

• You might write “W” when the subject wanders around.


To use this method, start a table with the elapsed time in one column and the action which occurs in the other.


By carefully noting what the subject does at each stage, you can locate the difficult parts.


Nondisclosure FormsRecording Permission

Informed Consent Form


Pre-test QuestionnairesInterviews


Task Scenarios


• Results they will strive to achieve

• Motives for performing the work

• Actual data/names rather than generalities

• State of system when task is initiated

• Any other materials they need to complete the task


be realisticprovide motivation


sequence tasks



Determine levels of interaction


The difference between strictly observing a person navigating a document or using a product and participating in what they do is significant.


But it is also a matter of degree– decide in advance just how much you will interact with your test subjects.



silent observation


talk-aloud protocols


Post-test Questions


• Major items related to specific research questions

• Items related to trends/patterns among participants

• Items related to things you can’t directly observe


Question Forms/Formats


Likert Scale


Semantic Differentials


Debriefing Guide


Technology

Usability Testing Materials