Upload
maaret-pyhaejaervi
View
397
Download
0
Embed Size (px)
Citation preview
1
Exploratory Testing Explained and Experienced
Maaret PyhäjärviEmail: <[email protected]> | Twitter: maaretp
Maaret PyhäjärviNimeä | Attribution (Finland)http://creativecommons.org/licenses/by/1.0/fi/http://creativecommons.org/licenses/by/1.0/fi/deed.en
2
TESTING IS ABOUT FEEDBACK:
QUALITY-RELATED INFORMATION
3
Realizations about Nature of Testing
2016
1639
5±2
4
20DYNAMICALLY ADAPT FOR LIMITED BUDGET
OPPORTUNITY COST
5
16 EXPECT THEUNEXPECTED
6
1639ROUTES ARE RELEVANT
NOT ALL BUGS ARE EQUAL
7
5±2TAKE NOTES
CREATE CHECKLISTS
8
Things Can Look Different from Different Perspectives
Exploratory Testing:Better tests, better testers!
• An approach, not a technique• Find unknown unknowns• Disciplined• Test is a performance, not
artifact– Artifacts support human
memory– Many forms: e.g. checklists
and automation• Exploratory performance
testing, Exploratory test automation, Exploratory regression testing
Test-related learning
Design of new tests
Test executionResult interpretation
9
10
There’s a Difference!• A comment I hear often on the ET
course I give:– ”I’ve always been doing this just did not
give it a name”• I must emphasize that:
– I require more out of exploratory testing than just ”going where you feel like while testing”
• Ask yourself:– Can you defend your choices of strategy
and tactics?– Can you explain what you’ve done in
words that don’t just explain numbers of bugs found?
– How do you know if you’re done or not?
”My testing would be
unsystematic ad hoc testing if I could not
tell the story of my tests, remember what I’ve
tested or what my strategy
was, or relate that back to
my mission” –James Bach
11
Exploratory Testing Self-Management
Vision (“Sandbox”) Current Charter
Other Charters Details
12
Exploratory Testing: Frame of Management
”A day’s work”
Vision (“Sandbox”) Current Charter
Other Charters Details
Bug Reports
Perception of quality
and coverage
Quality ReportDebriefin
g
Tester
Test Manager
PastResultsObstaclesOutlookFeelings
?#
xCharter backlog of the future testing
Out of budget
Next in importanc
e!#, ?, x, +20:20:60
Session sheets of the past testing
Idea of exploratio
n
Metrics summary
Coaching
PlaybooksCoverage outlines
14
Test Ideas / Quick-and-Dirty
Download the full 2-page Cheat Sheet with ideas from Elisabeth Hendrickson, James Lyndsay, and Dale Emery on Qualitytree.com
15
(Exploratory) Testing Dynamics Source: Adapted from James Bach, Jon Bach, Michael Bolton. Exploratory Testing
Dynamics. v.2.2. 2009
Evolving
work produc
ts
Skills and
tactics
Testing
polarities
Test strate
gy”A set of
considerations designed to help you
test robustly or evaluate someone
else’s testing.”
” To develop ideas or search a complex space quickly yet thoroughly, not only must you look at theworld from many points of view and perform many kinds of activities”
”Exploratory testing spirals upward
toward a complete and professional set
of test artifacts”
” …skills that comprise professional and cost effective exploration of technology. Each is distinctly observable and learnable, and each is necessary to exploratory work.”
16
Exploration SkillsSource: Adapted from James Bach, Jon Bach, Michael Bolton. Exploratory Testing
Dynamics. v.2.2. 2009
Self-manageme
nt
Developing ideas
Examining product
DoneTo DoIssuesCoverage
All sources available Best use of time – effective and efficient work
Making modelsTool support – creative solutionsRisk-based testing – scientific approachKeeping one’s eyes open
17
Test Automation / Tools in ET• Any form of acquiring quality-related
information fits into exploratory testing• It’s not manual, it’s brain-engaged – and for
making that happen, you need to be smart with automation and tools!
• Example: you might not at first know what you’re looking for...
Search "<ns1:Koodi>" (443 hits in 169 files) in VE Y:\ELLU\LAPA\Testaus\Kokonaiseläketurvan-ote\Ansaintatiedot-VE\HaeAnsaintatiedot-VE__ 20110307 14-22-15.xml (4 hits) Line 1313: <ns1:Koodi>YL130I</ns1:Koodi> Line 1317: <ns1:Koodi>LAPA_172_011</ns1:Koodi> Line 1321: <ns1:Koodi>67</ns1:Koodi> Line 1324: <ns1:Koodi>67</ns1:Koodi> A lot of text cut away from here…
18
Favoring Mindmaps (Mindmup, Xmind) and Notetaking (Rapid Reporter, iTester) tools
19
Testers don’t break your code, they
break your illusions about the code.
-- adapted from James Bach
20
Empirical Evidence over Speculation
VALUE
ILLUSIONS
•Code doing what it’s supposed to•Product doing what it would need to•Your process is able to deliver with change in mind•Business growing with uninformed risks on the product and the business model around it.
Serendipity, Perseverance …and Love of Testing
• Serendipity = Lucky accident
• Just my luck?– Luck favors the ones
who intentionally vary their actions
• ”The more I practice, the luckier I get”
• Perseverance = Keep trying
• Testing takes time – keep trying with more ideas, stop giving up so easily
• “It's not that I'm so smart, it's just that I stay with problems longer.”
22
The World Has Already Changed
COMMODITY TESTERS• Manual
checkers• Tests are an
artifact
SKILLED TESTERS• Explorers of
products and businesses
• Testing is a performance
Question, Thoughts?Feedback?