Upload
michael-powers
View
105
Download
2
Tags:
Embed Size (px)
DESCRIPTION
Done a usability study? Ready for the next step? Today we have an abundance of fast, affordable website user research methods, many of which can be done remotely with real users. Learn about available user research options and how IUP runs successful research projects that lead to actionable insights.
Citation preview
Fast, Cheap, and Actionable
Creating an Affordable User Research Program
Michael PowersDirector of Web Services
• museyroom.com
• twitter.com/mjpowers
• www.slideshare.net/michaelpowers
• www.linkedin.com/in/mjpowers
On the Web
What kinds ofuser research
have you done?
It all started with ausability test
Today• Getting Permission for Research
• A Process for User Research
• Inventory of Actionable User Research You Can Do Fast and Cheap
User Research Study↓
User Research Program
GettingPermission0
Don’t Ask
“We don’t want “a research“study.”
“A research study?
“Look, just make a red box on the home page, and put this text in there.
“I just need to make sure students don’t miss it.”
Nobody wants aresearch study.
Nobody wants a website.
What We Really Want• Applicants
• Successful Students
• Academic Research
• Donations
Create a Research Culture• Do a usability test and invite
stakeholders to watch
• Make research part of every project
• Apply previous research to new projects
• Show results early and often
Research Process1. Frame a Good
Question
2. Know What You Already Know
3. Choose Study Type
4. Find Participants
5. Run Study and Analyze Results
6. Take Action
7. Measure and Repeat
Frame aGood Question
1
Good questions are related to your goals
“Students can’t find stuff on our website.”
Focusing a QuestionHow can we decrease attrition?
• Can students find information about support services?
• Can students find out what’s happening on campus?
“The admissions website needs work.”
Focusing a QuestionHow can we get more/more qualified applicants?
1. Can prospectives find our list of majors? Our tuition information?
2. Can prospectives register for a visit?
3. Can prospectives successfully apply on line?
Goal-related questions lead to faster, cheaper, more actionable user research
Know What You Already Know
2
What Do You Already Know?• Best Practices
• Published Studies
• Data about Your Institution and Students
• Server Logs
• Analytics
• Past Studies
Get OrganizedQuantitative Data
KPI Spreadsheet
Qualitative Data
User Personas
KPI =KeyPerformanceIndicator
KPIs measure progress toward your goals.
KPI Spreadsheet• Focus on the KPIs you've
chosen
• Give specific reports to stakeholders who request them
• Track over time without running multiple tools
• Include anything numeric: survey results, 4Q survey, analytics, server logs, etc.
• Create specific analyses for specific projects
User Personas
Include ScenariosMotivation Scenario
Site Features Used Behaviors
Find the school with the right pre-Law program for me
Checks to see if her major is available
•Site Search
•Undergrad Admissions Website
•Academics Link
•Looks for “Pre-Law” major, does not find it.
User Personas• Fiction—but a useful
fiction
• Puts what you've learned into a format you can share with your team and clients
• If it is wrong, team or clients will let you know
• Helps bring new team members up to speed
Use these two tools to track your progress
over time.
Choose Study Type
3
If you’ve never done one, do a usability test first.
FocusGroups
Some Types of User Research• Usability Test
• Surveys
• Pop-up Survey
• KJ Session
• Card Sort
• Navigation Test
• Layout Test
• A/B Test
Some Types of User Research• Usability Test
• Surveys
• Pop-up Survey
• KJ Session
• Card Sort
• Navigation Test
• Layout Test
• A/B Test
Usability Test
Photo source: http://www.flickr.com/photos/l-i-n-k/3654390818/
Analytics
Survey
Pop-Up Survey
Pop-up SurveyWhat You Do: Ask site visitors questions about their
experience on a particular page or the whole website.
Good For: Capturing user intent; settling arguments about what belongs where
When To Use: When you have a question to answer, or all the time
Participants Needed: Calculate your sample size:http://www.surveysystem.com/sscalc.htm
Pop-up SurveyTools Available: •4Q Survey
•iPerceptions•CrowdScience•Roll your own
Cost: •$0–$10,000+
4Q Survey Questions1. Based on today's visit, how would you rate your site
experience overall?
2. Which of the following best describes the primary purpose of your visit?
3. Were you able to complete the purpose of your visit today?
4. Comment
• If yes: What do you value most about the website?
• If no: Please tell us why you were not able to fully complete the purpose of your visit today
“I am always able to find what I am looking for. The website is really easy to use.”
“I can never easily find what I'm looking for, if I can find it at all.”
Coding Comments• Visual Design
Positive
• Visual Design Negative
• Usability Positive
• Usability Negative
• Navigation Positive
• Navigation Negative
• Content Quality Positive
• Content Quality Negative
!" #!" $!" %!" &!" '!!" '#!" '$!" '%!" '&!"
()*+,)-./01+/2)3+4+56"
7)8"9./5:/5";"(::2"
<=:>+?>"=),:08+5:"
;@A"B:38+5:C"/./D9E<"
<+5:"<:)F>G"
9./5:/5"HI)4+56"
1+/2"A:.=4:"
@8)34:"
J+8I)4"K:8+,/""
(:L8")/2"M*:/58"
NF,)/+O)-./"
!"#$%&'()'*(##%+,-'.%+/(+0+1'230-'2(405'
2(405-'
637,'809':-%&-'*(##%+,';+<'
(:,)-*:"
A.8+-*:"
!"# $!"# %!"# &!"# '!"# (!"# )!"# *!"# +!"# ,!"# $!!"#
-./01.23456047.8090:;#
<.=#>34:?4:#@#-??7#
AB?C0DC#B.1?5=0:?#
@EF#G?8=0:?H#434I>JA#
A0:?#A?.KCL#
>34:?4:#MN.90:;#
6047#F?3B9?#
E=.89?#
O0=N.9#P?=014##
-?Q=#.47#R/?4:=#
SK1.40T.234#
!"#$"%#&'()*+,*-.()*"/#00*12&"
3#4'5&"
67'57"3#4'5&"8'9":&*;&"/#00*12"<#&2"%#&'()*=>+,*-.()*=>"?1@"
-?1.2/?#
F3=02/?#
Action• Changed our yearly goals to spend more time
on these issues
Card Sort
Photo Source: http://www.flickr.com/photos/rosenfeldmedia/3343501403/
Card SortWhat You Do: Give participants cards with website
topics on them, then ask them to put them into piles that make sense to them
Good For: Understanding the mental models of your users
When To Use: When creating or improving your information architecture/navigation
Participants Needed: 15 face to face50 online
Card SortTools Available: •Index cards
•OptimalSort (remote)•WebSort (remote)•xSort
Cost: $40-$600
Open• Users stack cards
then
• Give each pile a name
Closed• Give users header
cards• They assign cards to
the header cards
Where do we put counseling?
Action • Complete restructuring of the Student Life
section of our website
• Move counseling websites out of “Health” and into “Support”
One Subsite,Before and After
Change between Comparison Periods
Compared toWhole Site
Pageviews 76% 140%
Unique Pageviews 81% 142%
Bounce Rate -21% 27%
KJ Session
Photo Source: http://www.flickr.com/photos/drewm/370231088/
Navigation Test
Navigation TestWhat You Do: Give participants a proposed site
structure/navigation and let users drill down to find different things.
Good For: Verifying an IA in a more realistic way (not everything is visible); finding problem areas
When To Use: To verify a new information architecture
Participants Needed: 50
Navigation TestTools Available: •Treejack
Cost: $150–$700
Action• Rearranged Admissions Site navigation to
• Combine repeated material
• Put “Costs and Tuition” higher in the site tree
Layout Test
Layout TestWhat You Do: Participants respond to a static image
of your site
Good For: Behavior, Remote, Automated, Quant.
When To Use: Before building out a new design
Participants Needed: About 50 per test
Layout TestTools Available: •Paper
•PowerPoint•Five-Second Usability Test•Chalkmark•Usabila
Cost: $40–$600
Layout TestPositive Negative•Get reactions to real
designs without building out a whole site
•No interactivity
•Users can be confused when doing this remotely
A/B Test
Find Participants
4
The Right Number of Participants• Different methods have different requirements
• Plan to recruit more than you need
• But don’t test more than you need
The Right Kind of Participants• Steve Krug: “Recruit loosely and grade on a
curve”
• Good advice, but when possible try to recruit participants that are more like your users
• For instance: faculty and students see your website very differently
Finding Participants• People you know
• Students, faculty, and staff at your school
• Local high schools (talk to a guidance counselor)
• On your website (link on homepage or pop-up)
• Recruitment tools like Ethnio
Photo source: http://en.wikipedia.org/wiki/File:Ethnio_recruit_ad.png
Offer a prize
Run Study and Analyze Results
5
Keep track of results in KPI spreadsheet and personas
Take Action6
Share results and proposed solutions with stakeholders
Measure and Repeat
7
Share results
Challenge: Do some sort of user research each month
Research Process1. Frame a Good
Question
2. Know What You Already Know
3. Choose Study Type
4. Find Participants
5. Run Study and Analyze Results
6. Take Action
7. Measure and Repeat
More Informationmuseyroom.com
twitter.com/mjpowers
www.slideshare.net/michaelpowers