Embed Size (px)
DESCRIPTION
Test automation is software development. To automate tests well, you have to have brains. Unfortunately, the very brains that make you good at your job also make you highly attractive to zombies. Like all zombies, test automation zombies are brainless, insatiable, and relentless. Unlike human zombies, test automation zombies can be difficult to recognize. They don’t look like people at all. Some look like org charts. Some look like best practices. Some you can’t see at all—bad habits that “seemed like a good idea at the time.” Dale Emery has survived numerous test automation zombie attacks with almost no permanent damage. Dale knows zombies, and teaches you to recognize them. Automating only at the end of a sprint? Zombie. Sprinkling fixed delays through your tests to accommodate unresponsive web pages? Zombie. Automating only by record and playback? Zombie. Make no mistake—the test automation zombies are coming. Learn how to spot them before they eat your brains.
Citation preview
BT8 Session 6/6/2013 2:15 PM
“How to Survive the Coming Test Automation Zombie Apocalypse"
Presented by:
Dale Emery DHE
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073 888‐268‐8770 ∙ 904‐278‐0524 ∙ [email protected] ∙ www.sqe.com
Dale Emery DHE
Since 1980, Dale Emery has worked in both IT organizations and software product development companies as a developer, manager, process steward, trainer, and consultant. He helps people apply the agile values of communication, feedback, simplicity, courage, and respect to software development. Dale's combination of deep technical expertise and extensive organizational development experience makes him particularly effective in working with software teams. In 2007 Dale received the Ward Cunningham Gentle Voice of Reason Award, which the Agile Alliance created to recognize Dale’s unique contribution to the agile community. Dale's personal mission is to help people create joy, value, and meaning in their work. Learn more about Dale at dhemery.com.
How to Survive the Coming Test Automation
Zombie Apocalypse
Dale Emeryhttp://dhemery.com
@dhemery
Too gruesome to display
The Trouble With Zombies
LegionRelentlessInfectious
Eat your brains
Usually not as prettyas the one
on the previous slide
Test Automation ZombiesLegion
Relentless
Infectious
Eat your brains
Look just like ideas
Oddly attractive
The Trouble with Test Automation Zombies
Sometimes the best you can do
They are easy
Lull you into doing them by habit
Focus on the short term
At the expense of the (not-so-)long term
Pound FoolishZombies
The Record and Playback Zombie
AppealLots of tests quickly, (almost) for free
DangerRecorded tests are
unexpressive, brittle, unmaintainable
Can’t record testsuntil the software is done
The Automation is Easy Zombie
AppealInexpensive automators
DangerTests written by unskilled programmersare unexpressive, brittle, unmaintainable
Surviving Pound Foolish Zombies
Remember:Test automation is real software development
Test automation takes timeBig cost is maintenance
Writing maintainable code is hard
More Surviving Pound Foolish Zombies
Stop record and playback
Refactor recorded tests to make themexpressive, resilient, maintainable
Use real programmers...
with the experience to value maintainability...
and the programming skillto write maintainable test code
DisplacementZombies
The Automate Last ZombieAppeal
“Can’t automate tests before system is written”
DangerFeedback is delayed
The system is harder to test(not designed for testability)
Missed opportunityto use tests as guidance
The Dedicated Test Automator Zombie
AppealKeep developers focused on developing
DangerTest automation lags development
Feedback is delayedTypically uses less-skilled programmers
The Test Automation Group Zombie
AppealKeep entire development team
focused on developing
DangerSame as dedicated test automators
Organizational boundariesmake delays even worse
Surviving Displacement Zombies
Create automatable examplesbefore development begins
Developers automate testsDone includes test automation
MathZombies
The Test Case Count Zombie
AppealConfidence that the system is tested
DangerFocuses on ease of automating each test
Instead of on the value of the tests
The Code Coverage Zombie
AppealConfidence that the system is tested
DangerConfidence is unwarranted
(executed does not mean tested)
Surviving Math Zombies
Focus on automating teststhat provide value
Use code coverage toolsonly to identify what is not tested
StealthZombies
The Dedicated First Responder Zombie
AppealRelieves the development team from having
to run, maintain, respond to tests
DangerDelays feedback
(through first responders)
The Hide the Broken Feature Zombie
AppealReduce distraction by known failures
DangerReduces urgency to fix the features
(Disable tests for known-broken features)
The Hide the Flaky Test Zombie
AppealReduce wasted effort diagnosing tests
DangerDiscards potentially useful test results
Reduces urgency to make tests more robust
(Disable or ignore flaky tests)
The Automatic Rerun Zombie
AppealReduce wasted effort diagnosing tests
DangerDiscards potentially useful test results
Reduces motivation to make tests more robustIncreases suite run time
Surviving Stealth ZombiesFix flaky tests now
Fix broken features now
Bypass flaky technology
Analyze every failure
Include automated testsin code promotion procedures
Reserve automatic reruns for exceptional cases
HackerZombies
The Fixed Wait Zombie
AppealEasy way to deal with variable response times
DangerFixed waits always increase
Test suite execution timebecomes maximally pessimistic
Thread.sleep(8000);
Surviving The Fixed Wait Zombie
Bypass slow, variable technologies
Poll for the relevant condition
Arrange to be notifiedwhen the condition becomes satisfied
Create a utility for fixed waits
Instrument the utilityto gather information
The Dependency Chain Zombie
AppealSpeed up test suites
DangerPrevents running tests independently
Skips subsequent testseven if they would provide value
Harder to understandwhat responsibility each test is testing
Surviving The Dependency Chain Zombie
Make tests independent
Bypass slow technologiesfor setup and verification
Let each testestablish its own preconditions
The Predefined Test Database Zombie
AppealSpeed up test suites
Often already exists for manual testing
DangerHides essential details from test code
Harder to understand what responsibilityeach test is testing
Surviving The Predefined Test Database Zombie
Express every essential detaildirectly in the text of the test code
Let each testcreate its own data
Use the builder patternto create non-trivial test data
The Chewy GUI Zombie
AppealAvailable and accessible to automators
Tests whole system
DangerMakes tests dependent on most volatile interface
Increases test execution timeHarder to diagnose failures
(Testing only through the GUI)
SurvivingThe Chewy GUI Zombie
Bypass the GUI whenever it isnot essentialfor the test
How to Survive the Coming Test Automation
Zombie Apocalypse
Dale Emeryhttp://dhemery.com
@dhemery
Now You Know
