Upload
jen-mcginn
View
118
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Ana Chang and I presented this talk to PS SIG CHI on December 5, 2013. We also presented the material at Google and Chase on December 6th.
Citation preview
The RITE+Krug Method
Getting crazy good stakeholder involvement in Agile user research
Jen McGinn & Ana ChangDecember 5, 2013
We’ll tell you our story
We’ll listen to your stories
We’ll give you summaries and links to further reading
Hopefully, you’ll leave with something helpful
And we’ll pick up some tricks from you
Format
Jen McGinn and Ana Chang 2PS SIG CHI Monthly Meeting, Dec 2013
Before we tell you our story, let’s start with you
Show of hands:– How many of you are working with Agile development
teams?– How many of you are conducting user research (feedback
sessions) with those Agile teams?– How many of you are feeling really AWESOME about how
the dev team is involved in that research and that they make changes as a result?
Jen McGinn and Ana Chang 3PS SIG CHI Monthly Meeting, Dec 2013
I can’t believe he clicked the blue button.
I assumed he would click the gray button.
We should fix that!
Late 2011: Traditional usability tests were conducted
Jen McGinn and Ana Chang 4PS SIG CHI Monthly Meeting, Dec 2013
The results are not statistically significant.
This is not a typical user.
That’s not a realistic task.
Later 2011: The stakeholders didn’t buy into the results
I don’t agree with your interpretation of what that means.
Jen McGinn and Ana Chang 5PS SIG CHI Monthly Meeting, Dec 2013
Current state of the application
Start of study
Current state of the application
Proposed new design
At the beginning of the study
When the study results are published
The feedback came too late for our Agile environment
Jen McGinn and Ana Chang 6PS SIG CHI Monthly Meeting, Dec 2013
Now it’s your turn
• First names only• Agile User Research only• In less than a minute, tell us:
– What user research you are conducting now (method, frequency, # of participants)
– One thing that is AWESOME about thatOR– One thing that’s not working, that you wish you could fix
Jen McGinn and Ana Chang 7PS SIG CHI Monthly Meeting, Dec 2013
Hey, what about RITE testing?
Ana had attended a conference
Jen McGinn and Ana Chang 8PS SIG CHI Monthly Meeting, Dec 2013
Great – so now what?
Rapid Iterative Test Environment (RITE)– Developed by Dennis Wixon & the folks at Microsoft games– Can change the (live) system between participants– Developers must attend the UT sessions– Keep testing until show-stoppers are fixed– Number of participants and time to conduct undefined
Jen McGinn and Ana Chang 9PS SIG CHI Monthly Meeting, Dec 2013
Early 2012: What’s been written on Agile + RITE?
• When I started looking for answers 18+ months ago:• Lots and lots on Agile UX + design• Not so much on Agile UX + user feedback
• RITE (described in general terms, but no specifics)• Co-design • Customer/user “proxies”
• However, there were some elements of Steve Krug’s approach that stuck with me
Jen McGinn and Ana Chang 10PS SIG CHI Monthly Meeting, Dec 2013
Steve Krug … why does that name sound familiar?
http://www.amazon.com/Steve-Krug/e/B001KHCFUU/ 11
The Krug Approach*
“A morning a sprint, that’s all we ask.”
• Test 3 users in a morning
• Invite everyone to attend: product owners, developers, designers, writers, marketing …
• Debrief over lunch: 1 hour
– Only people who observed can attend/vote (sort of)
– Focus ruthlessly on most serious problems
• Report takes less than 30 minutes to write
– Just an email
– 2 page max., mostly bullet lists
* Originally presented by Steve Krug at the Boston UXPA 2013 Conference, May 201312PS SIG CHI Monthly Meeting, Dec 2013
Our RITE + Krug Combination
• Elements from both RITE and KRUG– Require that a stakeholder from dev, design and PM attend each
session– Can change the prototype between participants– 3 or 4 participants– Findings discussed in 1-hour debrief– Minimal “report” (bullets on a wiki page)
• Sessions went viral: 12+ observers• Could test every as often as every 2 weeks
http://www.upassoc.org/upa_publications/jus/2013may/JUS_McGinn_May_2013.pdfJen McGinn and Ana Chang 13PS SIG CHI Monthly Meeting, Dec 2013
Artifacts: scaled down script
Jen McGinn and Ana Chang 14PS SIG CHI Monthly Meeting, Dec 2013
Artifacts: example wiki page with schedule and impact
Jen McGinn and Ana Chang 15PS SIG CHI Monthly Meeting, Dec 2013
RITE+Krug Tips & Tricks
• Before sessions: Hold a kickoff meeting with stakeholders to get buy-in into the process and time required
• During: Keep a running list of observations and email them to the stakeholders within 30 minutes of the last participant
• After sessions: Ask attendees of the stakeholder meeting to review your list of observations prior to the meeting
• During debrief: Walk through the findings to get agreement on what everyone saw (optional: agree to next steps to fix)
• After debrief: Immediately document the observations and list of changes agreed to and send out/post on wiki
Jen McGinn and Ana Chang 16PS SIG CHI Monthly Meeting, Dec 2013
Let’s talk about Diana
The conversation has changed from ‘what is it?’ to ‘how can we master it and make it our own?’
- Diana DeMarco Brown (over lunch one day)
http://www.amazon.com/Agile-User-Experience-Design-Practitioners/dp/0124159532/
17PS SIG CHI Monthly Meeting, Dec 2013
Diana’s Agile Research Recommendations*
• Conduct research before the development cycle if possible– But if you can’t, fit it in anyway
• Predictable and repeatable user feedback sessions during the cycle• Complement user feedback sessions with fast, low-cost data
collection methods:– Surveys– Customer phone calls– Web analytics – Card sorts– Un-moderated usability tests
* Originally presented by Diana Brown at the Boston UXPA 2013 Conference, May 201318PS SIG CHI Monthly Meeting, Dec 2013
Jeff Gothelf’s Approach
• Continuous & collaborative– Continuous: every sprint– Collaborative: conduct
research together (dev, design, PM, &research)
Jen McGinn and Ana Chang 19
http://www.amazon.com/Lean-UX-Applying-Principles-Experience/dp/1449311652
PS SIG CHI Monthly Meeting, Dec 2013
Test everything
“In order to maintain a regular cadence of user testing, your team must adopt a “test everything” policy. Whatever is ready on testing day is what goes in front of the users… you’ll find yourself taking advantage of your weekly test sessions to get insight at every stage of design and development.”
– Lean UX, Jeff Gothelf
Jen McGinn and Ana Chang 20PS SIG CHI Monthly Meeting, Dec 2013
What I do now: Continuous and Collaborative
• Collaborative– Dev, Design, Doc, and PM stakeholders participate or observe– User researcher will facilitate or advise– Your opportunity to get “out of the building”
• Continuous– Sessions take place at regular intervals, say once a sprint– User researcher will recruit 3 – 4 target participants per session +
debrief same day
Jen McGinn and Ana Chang 21PS SIG CHI Monthly Meeting, Dec 2013
What else have people written on User Research + Agile?
Agile + RITE testing, finally (!) described in detail (2013):• “The RITE Way to Prototype”, Shirey, Charng, & Nguyen
http://uxmag.com/articles/the-rite-way-to-prototype
Agile + Cognitive walkthroughs (2013):• “Informal cognitive walkthroughs (ICW): paring down and pairing
up for an agile world”, Grigoreanu & Mohannahttp://dx.doi.org/10.1145/2470654.2466421
Agile + Wizard of Oz testing (2006): • “Adding Usability Testing to an Agile Project”, Meszaros & Aston
http://dx.doi.org/10.1109/AGILE.2006.5
Jen McGinn and Ana Chang 22PS SIG CHI Monthly Meeting, Dec 2013
Agile & Lean UX resources that include user research
• “Adapting Usability Investigations for Agile User-centered Design”, Sy (2007) http://www.upassoc.org/upa_publications/jus/2007may/agile-ucd.pdf
• “Successful User Experience in an Agile Enterprise Environment”, Federoff & Courage (2009)http://link.springer.com/chapter/10.1007%2F978-3-642-02556-3_27
• Jeff Gothelf’s blog: http://www.jeffgothelf.com/blog/
• 5 tips for applying Lean UX to User Research, Jeff Sauro (2013) http://www.measuringusability.com/blog/lean-research.php
Jen McGinn and Ana Chang 23PS SIG CHI Monthly Meeting, Dec 2013
Takeaways
• You and your team can incorporate user research into Agile development environments
• Align research with the Agile development & design cycles• Use lightweight research methods • Use a few participants• Require that key stakeholders attend (and/or conduct) user
research• Get feedback at regular intervals (once a sprint)• Get feedback on everything
Jen McGinn and Ana Chang 24PS SIG CHI Monthly Meeting, Dec 2013
Questions?
?
Jen McGinn and Ana Chang 25
jen dot mcginn at oracle dot com & ana dot ramirez dot chang at oracle dot com
PS SIG CHI Monthly Meeting, Dec 2013
The RITE+Krug Method
Getting crazy good stakeholder involvement in Agile user research
Jen McGinn & Ana ChangDecember 5, 2013
Different versions of the UI
Ease
of
Use
Large A/B study
Small, think-aloud study
Jen McGinn and Ana Chang
A/B testing
• A validation technique that you can use when you have:– Working code on the web– Large numbers of participants– A narrowly-defined problem
• “Which one of the designs results in fewer abandoned carts?”• “Which one of the designs results in a higher click-through rate?”
“Unless you are a huge property like – like Bing or Google … you’ll have more things to test than you have traffic
– Lean Analytics, Croll & Yaskovitz, Eric Reis, ed
Jen McGinn and Ana Chang 28PS SIG CHI Monthly Meeting, Dec 2013
“Formative” Feedback Sessions
• A discovery technique that you can use when you have:– drawings, sketches, wireframes– small numbers of participants– questions about features, mental models, terminology, iconography,
usability of your prototypes, current practices, pain points, values, design alternatives, or want to collect other qualitative data
“Initially, you’re looking for qualitative data … you’re speaking to people … You’re exploring. You’re getting out of the building.”
– Lean Analytics, Croll & Yaskovitz, Eric Reis, ed
Jen McGinn and Ana Chang 29PS SIG CHI Monthly Meeting, Dec 2013