Upload
jim-jarrett
View
501
Download
2
Tags:
Embed Size (px)
DESCRIPTION
90 minute training for experienced practitioners in best practices for analyzing and modeling qualitative user research, including KJ Analysis, personas, and scenarios. Tips and tricks and techniques included. Presented at the STC Summit 2010 on 3 May 2010.
Citation preview
Understanding Users Through Ethnography and Modeling
Jim JarrettSenior User Experience Architect and Manager
MedAssurant™
STC Technical Communication Summit 2010
3 May 2010
5:00-6:15pm
Reunion C
Understanding Users Through Ethnography and Modeling
Great product design starts with a deep understanding of the work that users do in the real world. Build your understanding through observation, interviews, surveys, and artifact collection. Share your understanding with KJ analysis, personas, and scenarios. Validate and prioritize your understanding with follow-up surveys.
Introduction and Conference Context 5 minutes
Ethnography and Data Collection 20 minutes
Modeling and Validation 30 minutes
Tips, Tricks, Tools, and Resources 10 minutes
Q&A 10 minutes
Who is Jim Jarrett?
Senior User Experience Architect and Manager
MedAssurant™
15 years at Rockwell Automationstarted as a multimedia developer in 1991other companies: Chiron, Roadway, IDD
Dozens of on-site observations in domains of medicalinformatics, industrial automation, and medical diagnostics.
Largest single user research project: 30 sites on 3 continents in 7 countriesacross 3 trips totaling 18 days on-site
Design for Six Sigma Blackbelt at Rockwell$2 million+ in hard savings over 2 years$24 million+ in revenue growth in same period
Multiple patents in UI for industrial automation
www.JarrettInteractionDesign.com
LinkedIn.com/in/jarrettinteractiondesign
Twitter.com/JarrettUX
immersed in context
Ethnography and Modeling
Sketching
Expert Inspections
Usability Evaluation
Usability Institute
“Understanding Users Through Ethnography and Modeling”Monday 5:00-6:15pm Reunion C
“Discovering Usability Defects Through Expert Inspections” with Rich GuntherTuesday 8:00-9:15am Landmark B
“Improving Product Design Through Usability Evaluation” with Scott ButlerTuesday 9:45-11:00am Reunion F
“Sketching User Experiences with the Design Studio Method” with Brian SullivanWednesday 8:00-9:15am Reunion B
Inception
Elaboration
Construction
Transition
“… truth will sooner come out of error than from confusion.”Sir Francis Bacon
“Better wrong than vague!”
“An articulated guess beats an unspoken assumption.”Fred Brooks
Define Ethnography
ethno folk, people, nationgraphy writing, study of
“The scientific description of nations or races of men, with their customs, habits, and points of difference.”OED
“The study and systematic recording of human cultures.”Merriam-Webster
Qualitative, language-based, empirical, descriptive research.
Techniques include observation, interviews, surveys, logs and journals, gathering of artifacts and images.
Holistic and contextual.
“I prefer design by experts - by people who know what they are doing.” Don Norman
“… pay attention to what users do, not what they say… ” Jakob Nielsen
“A computer shall not harm your work or, through inaction, allow your work to come to harm.” Jeff Raskin
“What do users do and how do they talk about it?” Jared Spool
Market Research vs. User Research
Market Research
Markets and segments
Best ways to influence
Target demographics
Buying behavior
Message definition
Focus groups, surveys, and interviews
Competitive analysis
“Why do people buy?”
User Research
Individuals and context
Best ways to work
User profiles
Goals and motives
Task definition
Observation, surveys, and interviews
Domain analysis
“What do people do?”
Data Collection Process
1. Define the target of your research. What questions are you trying to answer? Who should you study?
2. Arrange visits. Get permissions and release forms. Schedule timing for your targets. Coordinate with sales and marketing.
3. Prepare your site teams.4. Two sites at a time, two observers at a time. Introduce, observe, interview, wrap-up, thanks. Collect artifacts and notice context. 90 minutes.
5. Process with project team. Immerse them in the experience and data.
6. Follow up. Thanks + additional questions.
7. Retarget.8. More visits, but not too many!
Example Observation 1
“I hate this [expletive deleted] history screen! I have to jump all around and read everything to find out the patient’s latest A1C measurement and see if she’s visited the pedorthist recently.”
Option 1
“How can we improve the history screen?”
Option 2
“I noticed you came to this screen a couple times while you were talking to the patient. What were you looking for? Can you walk me through your thinking?”
Don’t ask users what they want, watch what they do.
Users are not designers, and you are not a user.
Example Observation 2
During the discussion with the patient, the nurse refers to a small spiral bound note pad from time to time, sometimes making a quick notation. She also jots some short notes on a piece of scrap paper. While you are paying attention to those activities, you miss some of the conversation with the patient.
Option 1“Can you repeat what you said to the patient? I missed some of the details.”
Option 2
“While you were talking with the patient, you looked at your notebook and sometimes wrote something. Other times, you wrote on the scrap paper. What were you writing down? Why in one place versus the other?”
Synthesize a big picture with rich, textured detail.Invest in more research, not post-processing.
Example Observation 3
BUT!
Get permission. Don’t violate policies, regulations, or laws.
Get a signed release form if necessary.
Don’t let the camera get in the way of really observing the work.
bunch of reference material printed from the web
support, clinic, and social worker numbers
calendar with both personal and work events
brand name vs generic reference for common drugs
SOPs and comprehensive clinic and phsycian contacts
today’s call list
monitor turned to allow photo; avoid HIPAA issues
ID badge worn at all times
follow up notes from last call
A picture is worth a thousand memory slips.
Triggers rich memories from visit, immediately afterward when processing the data as well as long term.
Provides rich, visceral context for team members and stakeholders who weren’t there.
Particularly memorable ones can be used for personas.
Data Collection Challenges
Confidentiality, privacy, and trust.
HIPAA, intellectual property, security, legal, regulatory.
Complex environments.
International sites, hazardous work, longitudinal or rare work.
Redundant data.
Repetitive work, rare exceptions, awkward times.
Too much data.
Many sites, trip logistics, team availability.
Exhaustion.
This is exhilarating work, but can be grueling.
Define Modeling
“A systematic description of an object or phenomenon that shares important characteristics with the object or phenomenon.”American Heritage Science Dictionary
“A simplified or idealized description or conception of a particular system, situation, or process… that is put forward as a basis for theoretical or empirical understanding… a conceptual or mental representation of something.”OED
Concrete and detailed representations of observations.
Purposefully lossy: generalized but retaining meaningful details.
Domain models include KJ Analysis, Personas, Scenarios and a variety of others.
“models can frame the design problem… help designers understand the domain” Jared Spool
“The most effective behavioral models are distilled from interview and observation data of real users into an archetypal description of how a particular type of person behaves and what their goals are.” Kim Goodwin
“A mental model is a picture of how your end users are supported by what you are creating.”Indi Young
KJ Analysis
“Affinity on steroids.”
Outcomes:
Shared understanding
Relationships
Priorities
Process steps:
1. Define the question to be asked of the data.
2. Reduce the data set to 30 or fewer items through multiple rounds of simple voting.
3. Group related items.
4. Title the groups.
5. Group related groups.
6. Create headlines for the higher level groups.
7. Visually lay out the groups with all data items.
8. Show relationships between groups.
9. Vote for the most important title‐level groups.
10. Draw a conclusion from the diagram.
We don’t have a clear definition of
what a requirement is
No clear definition of what
the difference levels of
requirements are
There seems to be
a discrepancy on
how detailed the
rqmts need to be.
I can’t delineate
between BR’s and
MR’s.
It’s not clear
where a market
requirement stops
and an SR starts
I don’t understand what the customer
needs; we don’t understand what is
important to deliver
I like to be involved in the whole process
All roles should be
involved throughout the
entire process
VOCs are very valuable
Is seems like we are
spending a lot of time putting
the same data in multiple
places and coordinating to
ensure the data is in sync
The process shouldn’t impede the
progress of the project
We’re going to
need to do some
clarification of what is
design and what is a
requirements
We don’t have a
good definition of
what a
requirement is
It seems like different
RAs have a different view
of what a requirement
should be
With requirements at
a high level it is difficult
to come up with a
good estimate
Perhaps we should
have two different
levels of use cases
We don’t know what “good
enough” is
How do you define
success at the end
of each iteration
We have lost the
concept of critical
release
requirements
We aren’t validating what
we are building with the
customer
Need to get out of the
lab exercise of I know
what the answer is now
I need to go find a
customer to tell me they
have that problem
Need to know when
we come out the other
end, have we built the
feature that is necessary
for the customer.
You better think about
performance
requirements before you
start down the design
path
Need a better way
to bring customer
feedback in earlier in
the process
VOCs are very
valuable for
providing focus to
product issues
VOCS were very
valuable which gave
us a very good
process and means
to rank and prioritize
requirements
Get engineering and
test involved earlier on
before the rqmts are
finalized
The requirement
analyst needs to
own the rqmts all the
way through the
process.
It’s difficult to look at
requirements in
ReqPro so everyone
gives up
I like the ability to
add attributes (Req
Pro)
Requirements tools are
difficult to work with
A lot of things you test
for will not show up in
requirements for
instance failure modes
If you read only the
SRs it is difficult to
understand what the
hell it is supposed to
do.
Hundreds of hours have been spend on
requirements with very specific detail for
consistency. The gap we have is that how
are we going to do a consistent design
without that level of detail
If the requirements don’t
have the detail I need,
where do I get them?
When we took UI detail
out of the use case it
wasn’t clear what the
rqmt really meant
Where do we put this
extra UI information
It’s nice to have implementation details
as an example to understand what the
RA was thinking
UI Information is a good
way to communicate ideas
Anything so that people
aren’t waiting for other
people to get stuff done
We should not have
to have everything
written down and
signed off to hand it
over the wall
No good way ot
fimplementing and
managing features access
products
We need to find a
better way to illustrate
requirements on a
features basis
All requirements
need to be
coordinated across
products
The process shouldn’t
impede the progress of
the project
To be honest, it’s to
painful to do a change
I don’t know where to go to get the information I need
Theme: Gaining a better understanding of the
requirement processes and how customers should be
involved
Lessons Learned: Better defections is needed for the
levels of requirements and where to get requirement
details,
Anatomy of a KJ Diagram
Headline
Relationshipcause and effect or contradiction
Title
Raw statement
Title with most votesTitle with second most votes Title with third most votes
Question to ask?
Summary answer.
Time/datePlaceParticipants
Personas
“The aspect of a person's character that is displayed to or perceived by others.”OED
Tells a story
Concrete and specific
Memorable
Day-in-the-life narrative
Goals, tasks, and other behavior
Skills and demographics
“Personas are helpful in creating and iterating a design, building consensus, marketing the product, and even prioritizing bug fixes.” Kim Goodwin
Persona: Identity
Memorable name with role or user type
Memorable quote that captures the essence of persona’s goals and attitude
Memorable photo that strongly represents persona and role
Persona: Narrative
Detailed and specific
Concrete
Summarize a typical work day.
Persona: Skills
Identify skills that differentiate the different personas.
Rate the persona’s skill level for each.
Present visually and simply.
Persona: Details
Extend, generalize, and add specifics to narrative.
Enumerated and prioritized.
More analytical and processed than narrative.
Persona Quality
P Primary researchIs the persona based on contextual interviews with real customers?
E EmpathyDoes the persona evoke empathy by including a name, a photograph and a product-relevant narrative?
R RealisticDoes the persona appear realistic to people who deal with customers day-to-day?
S SingularIs each persona unique, having little in common with other personas?
O ObjectivesDoes the persona include product-relevant high-level goals and include a quotation stating the key goal?
N NumberIs the number of personas small enough for the design team to remember the name of each one, with one of the personas identified as primary?
A ApplicableCan the development team use the persona as a practical tool to make design decisions?
http://www.userfocus.co.uk/articles/personas.html
Scenarios
“An outline or model of an expected or supposed sequence of events.”American Heritage Dictionary
Envision the future.
Personas drive the action and point of view.
Begin with a triggering event, describes a sequence of actions, and the results of those actions.
Technology and solution-free.
A few key scenarios are better than a comprehensive set.
“A scenario supplies the context-of-use necessary to set the stage on the subtlety and nuance you'll need to get the mix just right.” Jared Spool
Scenario ExampleFirst step is the trigger that begins the scenario
Step by step, who does what, when, and why
Call out questions or observations each step brings to mind
Scenario vs. Use Case vs. User Story
A scenario is a concrete, sequential narrative about a specific workflow.
A use case is a generalization of a number of scenarios and can generate multiple scenarios.
A user story is a single statement that captures who, what, and why. Usually names a use case or scenario, or is a single step in one.
“An outline or model of an expected or supposed sequence of events.”American Heritage Dictionary
“… a collection of possible sequences of interactions between the system under discussion and its external actors, related to a particular goal.” Alistair Cockburn
“A description of desired functionality told from the perspective of the user or customer.” Mike Cohn
Other Models
Contextual Inquiry Flow Models Sequence Models Artifact Models Cultural Models Physical ModelsOrganizational ModelsProcess ModelsStakeholder ModelsConcept Models
Choose models that communicate the most important information to the most important stakeholders.
Keep the raw data around for many purposes: Explain or justify the models. Future data mining. Explorable body of knowledge.
Validate the Models
Validate KJ priorities with surveys. Rate and rank.
Much larger audience.
Validate personas with interviews. “Which do you identify with?”
“What parts don’t work?”
“Describe a day in your work life.”
Iterate throughout the lifecycle and across multiple projects.
Over-communicate the models and advocate their use.
surveymonkey
Yahoo! personas poster from article at UIE.com
Tools, Tips, and Tricks of the Trade
Get really good at taking notes while watching and talking. Don’t use a laptop.
Don’t bother recording. Do another visit instead.
Exception: Livescribe Pulse Smartpen
Take lots of pictures (if permitted).
“Revolving Door” technique. Leah Rader and Beth Toland
Card sorting. Information Architecture
Controlled Vocabularies
Journaling.
What To Do with All This Research
Share it!Accessible repository, posters, walkthroughs with stakeholders, conversations with SMEs, users, and customers.
Invent new products.How could current work be transformed or eliminated?
Frame all downstream work.Use cases, UI designs, and test scenarios based on the real world.
Validate the system.Does it support real people doing real work? Does it help them achieve their goals?
Simplify the system.Eliminate functions and workflows that don’t support the work.
Discover new avenues.What more research paths are opened up?
Resources
The Design of Design: Essays from a Computer ScientistFrederick P. Brooks, Jr.
Contextual Design: Defining Customer-Centered SystemsHugh Beyer, Karen Holtzblatt
Rapid Contextual Design: A How-to Guide to Key Techniques for User-Centered DesignKaren Holtzblatt, Jessamyn Burns Wendell, Shelley Wood
Commercializing Great Products with Design for Six Sigma, chapter 16 “KJ Analysis”Randy C. Perry, David W. Bacon
Designing for the Digital Age: How to Create Human-Centered Products and Services Kim Goodwin, Alan Cooper
The Persona Lifecycle: Keeping People in Mind Throughout Product Design John Pruitt, Tamara Adlin
AgileProductDesign.comJeff Patton
Writing Effective Use CasesAlistair Cockburn