Embed Size (px)
DESCRIPTION
From the IXDA Boston night of Short Talks (http://tinyurl.com/4uoqhn) I had 10 minutes to give this 32-transition presentation: What\'s the Best Way to Compare Multiple Design Alternatives? Good design practice suggests you create multiple sketches, which eventually evolve into full-out design alternatives. However, once you have those alternatives, what\'s the best way to decide which one wins? I will review one approach and talk about the pitfalls and advantages to evaluating multiple designs at once. (What's even more amazing is that I was really busy that week and didn't start working on the presentation until I'd sat down in the audience to hear the other speakers. The entire presentation came together in under 90 minutes.)
Citation preview
Case Study:A Discount Approach to Comparing Multiple Design Alternatives
User Interface Engineering
Jared M. SpoolFounding Principal
©2008, User Interface Engineering. All rights reserved
The Problem
The Problem
Home page redesign for a big site4 million visitors a month
The Problem
Home page redesign for a big site4 million visitors a month
Never done a usability test before
The Problem
Home page redesign for a big site4 million visitors a month
Never done a usability test before
Only allotted 2 days of testing
The Problem
Home page redesign for a big site4 million visitors a month
Never done a usability test before
Only allotted 2 days of testing
Budget: less than $10,000
The Problem
Home page redesign for a big site4 million visitors a month
Never done a usability test before
Only allotted 2 days of testing
Budget: less than $10,000
5 design alternativesBy an outside design firm
Current Design
Current Design Alte
rnat
ives
#1
#2
#3
#4
#5
Current Design Alte
rnat
ives
#1
#2
#3
#4
#5
Evaluate in 2 days
Current Design Alte
rnat
ives
#1
#2
#3
#4
#5
Evaluate in 2 days
Under $10k
Approach #1:
“Within Subjects” with Multiple Designs
Learning effects
Too Many Combinations
Approach #2:
“Between Subjects” with One Design
Not enough users
but
but
Assumption Bias
Looking at the Problem Differently
Based on Inherent Value Testing
10 to 12 participants
Compares the existing design to the best alternative
Six-Step Process
Step 1:
Build a Weighted
Differences Matrix
DifferenceWeight Current
DesignDesign Alt #1
Design Alt #2
Design Alt #3
Design Alt #4
Design Alt #5
DifferenceWeight Current
DesignDesign Alt #1
Design Alt #2
Design Alt #3
Design Alt #4
Design Alt #5
Better search access
More descriptive links
Floating navigation
On-screen login
Visible top-rated items
8 2 3 1 5 3 5
6 3 4 2 2 3 1
5 2 5 0 5 3 0
4 5 5 3 3 3 1
6 1 3 5 3 5 0
DifferenceWeight Current
DesignDesign Alt #1
Design Alt #2
Design Alt #3
Design Alt #4
Design Alt #5
Better search access
More descriptive links
Floating navigation
On-screen login
Visible top-rated items
Total Scores
8 2 3 1 5 3 5
6 3 4 2 2 3 1
5 2 5 0 5 3 0
4 5 5 3 3 3 1
6 1 3 5 3 5 0
70 111 62 107 99 59
Step 2:
Recruit from Two User Groups
Step 3:
Use an Inherent Value
Testing Protocol
Step 4:
Add in “Best” Alternative
Step 5:
Add in Competitor’s
Design (when possible)
Step 6:
Evaluate Design
Alternatives against
Differences Matrix
DifferenceWeight Current
DesignDesign Alt #1
Design Alt #2
Design Alt #3
Design Alt #4
Design Alt #5
Better search access
More descriptive links
Floating navigation
On-screen login
Visible top-rated items
Total Scores
8 2 3 1 5 3 5
6 3 4 2 2 3 1
5 2 5 0 5 3 0
4 5 5 3 3 3 1
6 1 3 5 3 5 0
70 111 62 107 99 59
DifferenceWeight Current
DesignDesign Alt #1
Design Alt #2
Design Alt #3
Design Alt #4
Design Alt #5
Clearer trigger words
Interactive drill-down
Remember Previous Picks
Better search access
More descriptive links
Floating navigation
On-screen login
Visible top-rated items
Total Scores
9 0 1 1 3 3 1
10 0 2 2 1 4 5
10 0 0 0 5 0 0
8 2 3 1 5 3 5
6 3 4 2 2 3 1
5 2 5 0 5 3 0
4 5 5 3 3 3 1
6 1 3 5 3 5 0
70 140 91 194 166 118
Comparing Multiple Design Alternatives
Understand the differences between alternatives
Account for assumption bias
Focus on collecting data about the current design
Learn the problems that users are currently facing
Use the user data to make informed design decisions
More Info from User Interface Engineering
Newsletter: UIEtips (FREE)
UIE Virtual Seminars
Reports
User Interface Conference: www.uiconf.com
Blog: www.uie.com/brainsparks
Site: www.uie.com
