15 Tips For Effective Usability Testing In India

Usability Testing- An Introduction

&

Tips for Effective Usability Testing in India

By Abhay RautelaConeTrees.com , UXQuotes.com , TheUXBookmark.com , @conetrees

© Copyright 2010 Cone Trees2

Agenda

1. Introduction

• Usability?

• What is Usability Testing?

• Why conduct Usability Testing?

• When and What to perform Usability Testing on?

• How many users to test?

• The Usability Testing Process

2. Applicability

3. Tips for Effective Usability Testing in India

4. Questions & Feedback


What is Usability?

• Usability is what you may know as user friendliness- ease of use

• Usability is used in context of any object or a product a human

might interact with

• A product with good usability is one that is:

1. Effective- in helping the user accomplishing their goal for which they

are using the product

2. Efficient (can help the user accomplish their goals goal quickly)

3. Satisfying to use

4. Easy to approach and low error rate and criticality


What is Usability Testing

1. Usability Testing is a methodology

• To evaluate how easy a product is to use

• By giving tasks that are commonly performed or intended to be

performed on the product

• To people who are representative of the product’s target audience

2. Types

• Formative/ Exploratory

• Summative/ Assessment/ Verification or Validation

3. Towards the beginning of the PDLC, it’s good if you can conduct

usability tests on your competitions’ products


When and What to perform Usability Testing on

1. You can and should test at most phases of the PDLC/ SDLC:

• Requirements Gathering

• Design

• Development

• Testing

• Release


When and What to perform Usability Testing on

2. You can test from concept to ready product

• Prototype

• Level of detail

• Lo-fi to hi-fi (paper to interactive)

• Length and breadth of functionality

• Horizontal- all features, limited functionality

• Vertical- limited feature(s), full functionality within those feature(s)

• Product

• Section

• Complete


How many users?

• Research (Virzi, 1992, Nielsen & Landauer,1993) indicates that 5 users

uncover approximately 80% of a product’s usability problems

• Other research suggest different numbers

• The Truth: It depends

• A simple answer:

• For those interested in DIY usability testing: 5 is good to uncover an

appreciable number of usability issues in

• Increasing number of participants means significant increase in time and cost due to

increased complexity of planning, participant management and data analysis

amongst other factors

• Else consult with experienced professionals


Why conduct Usability Testing

1. In order to make a product user friendly you need to uncover

usability issues in it

2. You can’t do that on you own since:

1. You might not be the end user

2. Even if you belong to the target audience, you are probably not of the

same user expertise level as most of or a section of the users are.

3. You need to validate you assumptions

4. Make informed decisions. Have data by your side and make decisions

on facts, not hunches or opinions


The benefits of Usability Testing

1. Being penny wise, pound foolish does not pay. Don’t look at saving

a few days because of which you end up overshooting your plan by

months

2. Speed up development and cut costs by getting it right in the least

time and money possible by making it easy to use for the user and

designed in a way that best helps them accomplish their goals

3. PDLC integration: To continually identify and act upon usability

issues throughout the process at a low cost rather than trying to at

the end when it is more or less impossible


The Usability Testing Process

1. Plan

• Participant Recruitment

• Scenario/ Task creation

• Pilot Test

2. Execution/ Test sessions

3. Data Analysis

4. Report

• Checkpoint


Applicability

• Specifically applies to:

• Usability testing in India in organizations following an Agile SDLC

• More towards participants belonging to or residing in metros

• You should be interested if you are in India and:

• Are interested in learning about usability testing or are a usability engineer

• A product manager interested in learning about usability testing

• A start up interested in how usability testing can benefit your product

• Even if you are outside India:

• You should still be able to derive value from the tips if you are conducting

usability tests in countries outside India


Applicability of Usability Testing Tips

• Why tips on India are different from the books?

• India has a different cultural system as compared to the west

• Culture

• Value system

• Language

• Ways of working and interfacing with people

• Hofstede’s cultural dimensions

• There is a distinction between the working of Indian internet companies

and multinational corporations or foreign firms


Tip 1 (Planning) Create a Plan that Will be Read

• Suggest plan formats to the stakeholders

• Finally, create a plan according to their preference


Tip 2 (Planning) Participant Compensation that Works

• Kids- Toys, Chocolates, Confectionary

• School- Movie tickets, Cash

• College- Cash, Movie tickets, Shoppers Stop/ Life Style Coupons, Big Bazaar coupons

• Working Professionals- Lunch/ Dinner coupons, Shoppers Stop/ Life Style Coupons, Big Bazaar coupons

• Elders- Big Bazaar coupons

• Recruitment during festival season is tough. Reverse the flow by offering festival related gifts as compensation instead of the usual


Tip 3 (Recruitment) Talk like the User

• When calling prospective participants over the phone, speak in the language you feel the user is comfortable in, and can communicate and understand better to increase your chances of recruitment.

• When the person at the other end is comfortable with you, and more importantly gets some feeling of trust, the better your chances or successful recruitment

• Mix language and use slang when appropriate

• Empathize with all users

• Affection with children and respect to the elderly is required and just as beneficial


Tip 4 (Execution) Record using Inbuilt Laptop Camera

• Inbuilt laptop cameras are hardly noticeable*

• Reduce awkwardness and intimidation for the camera conscious,

especially effective with participants from non-metros

• Decrease introduction of bias, increase accuracy of test results

* The user must be briefed about recording them. Their permission is required before

proceeding. Get a CDA/ NDA signed.


Tip 5 (Execution) Simulate as much as possible

• Use a bandwidth shaper

• E.g.- You might want to limit speed to 56 kbps if this is the internet

speed available to most of your users rather than using high speed

internet connectivity available at your office

• Same goes for OS, browsers and resolution


Tip 6 (Execution) Use ‘Think Aloud’ only if Appropriate

• Use ‘Think Aloud’ protocol only if participants are able to adapt to it

easily,

• Feels unnatural for many thus increasing bias

• Adds to the pressure and increases bias, may affect performance

• Retrospective evaluation is an alternative but is time consuming


Tip 7 (Execution) Speak & Dress like the participant

• Try and dress like your participants, don’t be intimidating

• Speak in the language you feel the participant is comfortable in, and can

communicate and understand better to increase your chances of

recruitment

• This will help decrease bias introduced due to intimidation or

awkwardness

• The more comfortable the participant, the more your chances of accurate

results

• Caution! Don’t hand the reins over to the user


Tip 8 (Execution) Stakeholders as observers

• Let the stakeholders get a first hand experience of how the user actually uses the product

• Observation by stakeholders will help increases transparency and their belief validity of a methodology they are new to

• If they can not be observers, provide access to all the video recordings of the sessions or even better, a highlights video

• Stakeholders observing participants is more influential in them understanding usability issues rather than handing out a report


Tip 9 (Execution) Usability testing at Elders homes

• In the field, try to restrict team to 2 members

• One facilitator and one observer is good enough

• Let them know how many will be coming in advance

• Show your ID/ business card- Most elders are not comfortable with

the idea of strangers entering their homes


Tip 10 (Reporting) Prioritization also by LOE

• Prioritize by criticality and frequency of issues so stakeholders can

decide on incorporation as required

• When time and resources are scarce, stakeholders will pick

recommendations requiring least effort to implement


Tip 11 (Reporting) Provide quotes in original language

• Don’t translate quotes in English. When you translate a quote, you

loose quality of the insight gained through it. (Think JPG- lossy)

• You can provide a translation (in the rarer case) that you have

stakeholders who don’t understand the language the quote is in


Tip 12 (Reporting) Create a Report that will be read

• Discuss in advance the format of the report with stakeholders

offering suggestions to them on it

• Don’t deliver the report in a format that the stakeholder won’t read

• Avoid an unpleasant surprise with a format that contains or misses

out on information (and sorting) that the stakeholders wanted


Tip 13 (Reporting) Present Report

• Submitting it does not guarantee it being read or stakeholders

acting upon it

• Always try and present the report

• To ensure comprehension of the entire report to stakeholders

• To understand why certain recommendations can not be

incorporated

• To prevent information distortion


Tip 14- Checkpoint

• It is not uncommon in an Agile SDLC to see reports disappear over

time without any incorporation of agreed recommendations

• Checkpoint at appropriate intervals of time to keep track of and

follow through on recommendations agreed to be incorporated


Tip 15: Overall Process

• Involve stakeholders/ Product Department

• Absolutely important if usability is just beginning to get recognized in

your organization or you are part of the few who are part of the group

busy evangelizing

• A must if you are at

• Stage 2 to 5 of Nielsen’s Corporate Usability Maturity

• Why?

• They make or break usability within the organization

• Ultimately, they give the nod, allocate resources and budget for usability

• If you shut doors on them, they can shut you down since you are

probably conducting usability tests because they are allowing you to


Questions?


Feedback


Thank you

Website: http://www.conetrees.com Email: hello at conetrees dot comTwitter: http://www.twitter.com/conetreesUseful Quotes on User Experience: http://www.uxquotes.com The UX Practitioners’ resource: http://www.theuxbookmark.com


Join the UX Book Club, New Delhi http://uxbookclub.org/doku.php?id=new_delhi

As creator and lead coordinator of the UX Book Club, New Delhi, I’d like you to consider joining us if you are work in user experience in and around New Delhi

Join the SlideShare Accessibility Group slideshare.com/group/web-accessibility

View and share useful presentations on web accessibility- Join the group

Technology

15 Tips For Effective Usability Testing In India