Making sense of the data

Making sense of the dataCollaborative techniques for analyzing observations

Dana Chisnell@danachis#makingsense

1

What I’m talking aboutIn-time reporting, collaboratively

Shortcut to persona attributes

Ending the opinion wars through team analysis

Democratic, fast prioritizing

2

Rolling Issues Lists

Rolling issues listsObservations in real time

Observer participation = buy-in

Low-fi reporting

Observations in real timeLarge whiteboard

Colored markers

Don’t worry about order

Be clear enough to remember what was meant

Rolling issues

Rolling issuesAngela Coulter

Observer participationAfter 1-3 participants, longer break

You start

Invite observers to add and track items

Weighting

Number of incidents

Who’s who

Consensus: Observers buy in

observers contribute to identifying issuesyou learn constraintsinstant reporting

Big ideas, so far

A3 Persona studio

MatthewAttorney House in Tel Aviv MarriedLoves his BlackberryHates emailReads NYTimes on iPadAddicted to Words With Friends

!

Buying tickets on a website for for the Maccabiah

EdithFollows American sports, especially baseballHates ESPN.com Loves face-to-face online

DennisBuilding contractor Cell phone is for calling home Very tuned in to current eventsSees no usefulness in FacebookRecently more absent-mindedDifficulty doing standard calculations

Jane1,000-acre farm Tweets soil chemical readings Tablet instrumented to aggregate data

Personas: Archetypal users‣ Composites of real users

‣ Function or task-based

You: designer & developerHow do you design for these people based on what you know?

Think about each of the people as you answer these questions:

What can we say about how persistent the person is?

How pro-active will this person be in solving problems?

How easily will this person become frustrated?

How tech savvy is this person?

How literate is this person in the domain?

Little data, lots of assumptions

MatthewBlackberry-loving attorney

How old are they?

54 83 28 60

EdithESPN.com-hating Hangouts fan

DennisFacebookwhat?

JaneFarmer nerd

How old are they? How educated are they?

How much money do they make?

These don’t matter.

If demographics don’t matter, what do you do?

• persistence with tech

• tolerance for risk & experimentation

• how patient, how easily frustrated

• tech savviness, expertise

• strength of tech vocabulary

• physical or cognitive abilities

Ask the right questions

•Attitudemotivation, emotion, risk tolerance, persistence, optimism or pessimism

•Aptitudecurrent knowledge, ability to make inferences, expertise

•Ability physical and cognitive attributes

!

A3

Technique‣ Focus on the attributes that matter

‣ Accounting for what the user brings to the design

‣ Adjustable to context, relationships, domain

Tool‣ Framework to think about provisional or proto-

personas (little or no data)

‣ Schema to evaluate existing personas’ strength of coverage

Collaboration tool‣ Do our personas cover all the right attributes?

‣ Have we heard from all the users we need to hear from?

Task + functionality!

Buying tickets on a website for the MaccabiahVIP?

Suites?

Close to participants?

Event?

• Who is the most persistent when it comes to working with technology?

• Who is the most likely to experiment and create workarounds when something doesn’t work the way they expect?

• Who do you think will give up when they encounter frustrations out of impatience?

Ask the right questions

• Which one has the most tech expertise? Which one strikes you as the least tech savvy?

• Which one is the most likely to call tech support to help them get out of some tech pickle?

• Who will have the most advanced tech vocabulary when they do call for support?

Ask the right questions

MatthewAttorney House in Tel Aviv, marriedAssistant books reservations Loves his BlackberryHates emailReads NYTimes on iPadAddicted to Words With FriendsDoesn’t spend a lot of time on the Web Avid hiker and birderBad knees Needs Rx eyepiece for scope

!

Where does Matthew fit?

M

M

M

EdithAvid sports fanFollows Detroit Tigers Hates ESPN.com Loves face-to-face onlinePicked up Skype early, quickly Prefers Google Hangouts Dropped FaceTime - gestures were frustrating

!

Where does Edith fit?

E

E

E

DennisBuilding contractor Married to a nurseThey have 2 kids8-year-old cell phone Gets current events on the Web Sees no usefulness in FacebookTrouble sleeping, easily distractedServed in the military: Blunt Force Brain Trauma

!

Where does Dennis fit?

D

D

D

Jane1,000-acre farm 12 different systems every dayTweets soil chemical readings Smartphone alerts from exchangesTablet instrumented to aggregate dataMinimized manual inputArthritic thumbsNeeds stronger progressive lenses

!

Where does Jane fit?

J

J

J

!

A3 Persona modeling

M

M

M

E

E

E

D

D

D

J

JJ

How do design decisions change with these attributes?

Subtlety of affordances

Size of targets

Directness of the happy path

Feedback modalities

Labeling & trigger words

Amount of copy

Wording of instructions & messages

If you had this tool, what would you do next?What would be different for your design?

• Designing for attitude, aptitude, & ability

• Accounting for what the user brings to the design

• Checking that personas cover the right things

A3 Persona modeler

Persona modeler: fast way to visualize users by asking important questions

framework for talking about who users are

can tell which users might be missing

works with any amount of data

middle ground between demographics and research-based personas

Big ideas

What obstacles do teams facein delivering the best possible user experiences?

Guess the reason

Observation to Direction

Observations to direction

Observation Inference Direction

Observations

Participants typed in the top chat area rather than the bottom area.

ObservationsSources:

usability testing

user research

sales feedback

support calls and emails

training

What you saw

What you heard

Inferences

- Participants are drawn to open areas when they are trying to communicate with other attendees - Participants are drawn to the first open area they see

InferencesJudgements

Conclusions

Guesses

Intuition

+ Weight of evidence

Inferences Review the inferences

What are the causes?

How likely is this inference to be the cause?

How often did the observation happen?

Are there any patterns in what kinds of users had issues?

Direction

- Make the response area smaller until it has content

DirectionWhat’s the evidence for a design change?

What does the strength of the cause suggest about a solution?

Test theories

Big ideas: Making sense of the data

Observation What happened

Inference Why there’s a gap between behavior & UI

Direction A theory about what to do

KJ Analysis

ReviewHow’d that go? What might be different for your situation?

Priorities, democratically

reach consensus from subjective data

similar to affinity diagramming

invented by Jiro Kawakita

objective, quick

8 simple steps

1. Focus questionWhat needs to be fixed in Product X to improve the user experience?(observations, data)

What obstacles do teams face in implementing UX practices? (opinion)

2. Organize the groupCall together everyone concerned

For user research, only those who observed

Typically takes an hour

3. Put opinions or data on For a usability test, ask for observations

(not inferences, not opinion)

No discussion

4. Put notes on a wallRandom

Read others’

Add items

No discussion

5. Group similar items In another part of the room

Start with 2 items that seem like they belong together

Place them together, one below the other

Move all stickies

Review and move among groups

Every item in a group

No discussion

6. Name each groupUse a different color

Each person gives each group a name

Names must be noun clusters

Split groups

Join groups

Everyone must give every group a name

No discussion

7. Vote for the most important Democratic sharing of opinions

Independent of influence or coercion

Each person writes their top 3

Rank the choices from most to least important

Record votes on the group sticky

No discussion

8. Rank the groupsPull notes with votes

Order by the number of votes

Read off the groups

Discuss combining groups

Agreement must be unanimous

Add combined groups’ votes together

Stop at 3-5 clear top priorities

Goal: share the most important observations

Focus: identify priorities

democraticquick

Big ideas, so far

Big idea: Collaborative analysis

build consensus in real time: rolling issues observers contribute to identifying issues you learn constraints instant reporting

model users: A3 persona modeler

attitude

aptitude

ability

make sense of the data, together

observation: what you heard, saw

inference: why the gap between the UI and the behavior

direction: a theory about what to do about it

No reports. :)

Where to learn more

Dana’s blog: http://usabilitytesting.wordpress.com

Download templates, examples, and links to other resources from www.wiley.com/go/usabilitytesting

dana@usabilityworks.net

www.usabilityworks.net 415.519.1148

@danachis

Dana Chisnell