Embed Size (px)
Citation preview
Real Exploratory Testing, Now with a SOA Twist
Copyright 2009 Sogeti Denmark A/S. All rights reserved.
Opening
Wasting time …
Introduction
New Exploratory Testing approaches- Hotspot- Coffee Break
Exploratory Testing in SOA- Time optimization- Find more defects
Presentation
Christian Carlsen – Test Consultant
Michael R. Borlund – Test Manager
Test perspective in an SOA Environment
Service 1 Service 2 Service 3
Service 4 Service 5
Service BusService Bus
Service Test
Service Integration Test
End-to-end Test
Business Driven Test Management (BDTM)
Concept- Business alignment- Based on risk analysis- Limited flexibility (time gaps)
Steps of the BDTM test process
Exploratory Testing “hotspot” process
Concept- Categorization by defects- Navigators- Impact analysis
Exploratory Testing Coffee Break process
Concept- Initial meeting- Coffee analysis- Value analysis
Resumé
Minimize the wasting of time!- Use ET as creative testing
Benefits- Find more defects- Add value to the project
Concerns- Analysing data- Guiding and coordination
Opening
Results
Results
14 percent more defects found
Broader coverage
Additional knowledge of the system tested
Overall results
“Hotspot” vs. Coffee Break
”hot spot” Coffee Break
Total observations 3 10
Total defects found 11 8
Average defects found per observation
3.7 0.8
Variance of defects found per observation
4.3 0.6
Average time to find a defect (min) 33 19
Level of documentation high low
Knowledge of the system tested high low
Coverage broadness
Theoretical coverage broadness
Discussion (1/2)
Discussion- Structured management - Scarce test observations- High data uncertainty and variance
“Hotspot”- Obtain additional knowledge- Finding “unstructured” defects
Coffee Break- Finding “ low documented” defects- Learning about the system
Discussion (2/2)
Advise and solutions- Use both methods by own customized mix - Very short turn around time with these methods
Issues and difficulties- To accept the use of Exploratory Testing- Lack of comprehensive observations
Benefit and Successes- Finding different defects and a broader coverage- Obtaining knowledge and good for on boarding
When to use?
- Structured Test Management- Time gaps- Obtain knowledge
Conclusion
Finding 14% more defects
Better use of time
Broader test coverage
Final words
There's a first time for everything!
"Twenty years from now you will be more disappointed by the things you didn't do
than by the ones you did do. So throw off the bowlines. Sail away from the safe
harbor. Catch the trade winds in your sails. Explore. Dream. Discover.“
- Mark Twain
