Customer Validation using Diary Studies
LeanCamp London 2
January 18, 2012
@boonych • #leandiary
About me
UX designer at SapientNitro
Formerly a developer (startups, prototyping, telecoms, rich internet apps)
MSc Human-Computer Interaction
UXCampLondon organiser
@boonych • #leandiary
uxcamplondon.org
Why this talk
• Lean startup “epiphany”
• Gaps: Need more qualitative methods for holistic view
• Applying & integrating
• Learning & sharing
@boonych • #leandiary
The point
Plan and visualize with _validated_ data
Improve customer understanding and decision making process
Improve chances of building what customers really want
@boonych • #leandiary
Some assumptions
@boonych • #leandiary
“The goal of a startup is to figure out the right thing to build—the thing customers want and will pay for—as quickly as possible.”
Eric Ries, The Lean Startup
@boonych • #leandiary
“The goal of a startup is to figure out the right thing to build—the thing customers want and will pay for—as quickly as possible.”
Eric Ries, The Lean Startup
@boonych • #leandiary
Validated customer insight
Actionable learning, tightly
integratedwith tools &
metrics
“…customer archetype is a hypothesis, not a fact. The customer profile should be considered provisional until the strategy has shown via validated learning that we can serve this type of customer in a sustainable way.”
Eric Ries, The Lean Startup
@boonych • #leandiary
“…customer archetype is a hypothesis, not a fact. The customer profile should be considered provisional until the strategy has shown via validated learning that we can serve this type of customer in a sustainable way.”
Eric Ries, The Lean Startup
@boonych • #leandiary
Continuous researchImprove research methods iteratively
Evolve the data & learning
Customer Validation
@boonych • #leandiary
Diary Studies help here
Caveats
• Only the basics
• Not a silver bullet
• There is help around
• Learn from each other
@boonych • #leandiary
Diary Studies from 30,000 feetWhat? How? Why?
Analyzing and gaining insights
Practical tips
Ideas on how to apply
@boonych • #leandiary
A diary study involves participants reporting their activities over a specific period of time, usually apart from the researcher and in their normal daily lives.
@boonych • #leandiary
Benefits
• Why users behave a certain way
• Right context & environment
• Remote
• Sample over a duration
• Bridge qualitative & quantitative
• Integrate with metrics & tools
• Flexibility
@boonych • #leandiary
@boonych • #leandiary
Users record thoughts, comments, etc. over time
http://www.flickr.com/photos/vanessabertozzi/877910821http://www.flickr.com/photos/yourdon/3599753183/http://www.flickr.com/photos/stevendepolo/3020452399/http://www.flickr.com/photos/jevnin/390234217/
Interview usersGather feedback, data
Organise and analyse(affinity maps, analytics)
Participants keep a record of“When” data
Date & time
Duration
Activity / task
“What" data
Activity / task
Feelings / mood
Environment / setting
@boonych • #leandiary
No one right way to collect data
@boonych • #leandiary
Structured
Yes/no
Select a category
Date & time
Multiple choice
Unstructured
Open-ended
Opinions / thoughts / feelings
Notes / comments
Combine / mix & match
http://www.flickr.com/photos/roboppy/9625780/http://www.flickr.com/photos/vanessabertozzi/877910821
Organize before analyze
@boonych • #leandiary
Affinity mapping (integrate with “To the whiteboard”* exercises)
•Extract notes from interview data•Place similar notes together, repeat•“tag” clusters according to topic, title, concept, etc.•Extract learning and insights by illustrating the whole concept•See “Rich Pictures” from Soft Systems Methodology
*See “Enterprise guide to customer development” by Brant Cooper & Patrick Vlaskovits
Demo time!How to make sense of your data
@boonych • #leandiary
“Hygiene” aspects
At the beginning
•Introduction / get-to-know-you
•Demographics & psychographics, profiling
•Instructions / Setting expectations
At the end
•Follow-up
•Thanks / token gift
•Reflection
@boonych • #leandiary
Pitfalls
• Belief bias
• Behavior adjustment
• Ramp-up time
• Failure to recall
@boonych • #leandiary
Ideas!
@boonych • #leandiaryhttp://www.flickr.com/photos/danieloop/5336396524/
Start small, be nimble
• Low-tech (email, SMS, twitter, paper/pen, phone interviews)
• Simple, not complex, diary forms
• Let users just ‘dump’ facts, don’t force them to think so much
• Catch up regularly & share often
• Tweak on the fly
@boonych • #leandiary
Bring an observer or two
• Efficiency is shared experience
• Bring one or two key stakeholders along to the interview sessions (e.g. lead dev, mktg, sales, CEO, etc.
• Share and contrast: Don’t be precious about your own viewpoints
@boonych • #leandiaryhttp://www.flickr.com/photos/qkgirl/4837119913/
Map diary to session tracking
@boonych • #leandiary
Validate diary data against your metrics. Customers sometimes behave differently than what they say. Make the data richer.
Integrate with business metrics
• Use diary studies to test business assumptions
• Test against segments
• Useful for generative or evaluative research
• Scale up or down
• Test with prototypes
• Etc.
@boonych • #leandiaryTaken from “Enterprise guide to customer development” by Brant Cooper & Patrick Vlaskovits
‘Hidden’ insights from qualitative data
• Useful for new ideas
• Generates more questions
• Hidden facts about customers
• Stuff you never knew to look
• Store them for later
@boonych • #leandiary
Take-aways
• Diary studies are flexible, tweak to your requirements
• Use it to address gaps in your research data
• Integrate it with your existing measurement machinery
• Refine and iterate your approach
@boonych • #leandiary
ThanksMore on diary studies
http://www.webcredible.co.uk/user-friendly-resources/web-usability/diary-study-guide.shtml
http://www.system-concepts.com/articles/usability-articles/2011/a-quick-start-guide-to-online-diary-studies.html
Analyzing using affinity maps
http://www.usabilitybok.org/methods/affinity-diagram?section=how-to
http://www.mindtools.com/pages/article/newTMC_86.htm
Gamestorming book by Dave Gray, Sunni Brown, James Macanufo
Get in touch: @boonych on Twitter, or gluethink.com
@boonych • #leandiary