Upload
hakhuong
View
217
Download
1
Embed Size (px)
Citation preview
Beginning with the End in Mind: Driving Development with Acceptance Tests Elisabeth Hendrickson
Quality Tree Software, Inc.
www.qualitytree.com
Last updated November 10, 2009
This work is licensed under the Creative Commons Attribution 3.0 United States License. View a copy of this license.
What is ATDD?
Copyr
ight
© 2
01
0 Q
ual
ity
Tre
e So
ftw
are,
Inc.
Acceptance-Test Driven Development (ATDD) Cycle
!"#$%"&
'"(
)&""*+#",*
!$-./--0&1(/.2
0&1(/.23,.4#15
-21&6-21&6-21&6-21&6-21&6
!"%"#17
(Model developed with Pekka Klärck, Bas Vodde, and Craig Larman.)
Copyr
ight
© 2
01
0 Q
ual
ity
Tre
e So
ftw
are,
Inc.
ATDD: Discuss
!"#$%&'()&*+#,
-'#".-'#".-'#".-'#".-'#".
!"#$%&'(&%$)*#(&+(,
Concrete Examples with Expectations
(Slide developed in collaboration with Pekka Klärck.)
Copyr
ight
© 2
01
0 Q
ual
ity
Tre
e So
ftw
are,
Inc.
ATDD: Develop
!"#
$%""&'
(")& !"#$
%#$&
'()*+"&
%#$&
'*&+%","-."/,/
,-(*
.&/*/
(Slide developed in collaboration with Pekka Klärck.)
Copyr
ight
© 2
01
0 Q
ual
ity
Tre
e So
ftw
are,
Inc.
ATDD: Deliver
FeatureFeature Feature
Feature
Feature
FeatureFeature
Feature
Feature
Product
(Slide developed in collaboration with Pekka Klärck.)
Introducing an Example
Copyr
ight
© 2
01
0 Q
ual
ity
Tre
e So
ftw
are,
Inc.
Start with the Story
As an administrator, I want users creating accounts to be required to choose secure passwords so that their accounts cannot be hacked by someone using a password guessing program.
Copyr
ight
© 2
01
0 Q
ual
ity
Tre
e So
ftw
are,
Inc.
Discuss
What does “secure” mean
to you? At least 6 characters with
at least one letter, one symbol, and one number.
And if a user provides an insecure password,
display an error message.
Who’s in the room?
Product Owner, Testers,
Developers, and anyone else who will
touch the story.
Copyr
ight
© 2
01
0 Q
ual
ity
Tre
e So
ftw
are,
Inc.
Capture Concrete Expectations and Examples
Password Valid? “p@ssw0rd” Yes “p@s5” No “passw0rd” No “p@ssword” No “@#$%1234” No
Given a user is creating an account
When they specify an insecure password
Then they see a message, “Passwords must be at least 6 characters long with at least one letter, one number, and one symbol.”
Or can be expressed in tables
Can be expressed as
“Given - When - Then”
Or in other formats depending on the
Framework
Why ATDD? Reason #1: Drive Out Ambiguity and Clarify Expectations
Copyr
ight
© 2
01
0 Q
ual
ity
Tre
e So
ftw
are,
Inc.
This is not an Argument about a Bug
“Bug Triage Meeting” The Tuesday before release.
Whether or not it’s a bug, if we make a change we’ll blow the
schedule.
It’s a bug.
No it’s not.
Is too.
IS NOT.
IS TOO!
NOT NOT NOT!
Copyr
ight
© 2
01
0 Q
ual
ity
Tre
e So
ftw
are,
Inc.
Acceptance Tests Define Scope
A Short Digression on ATDD-Friendly Tools
Copyr
ight
© 2
01
0 Q
ual
ity
Tre
e So
ftw
are,
Inc.
Examples of ATDD-Friendly Frameworks
• Cucumber: a Ruby-based BDD tool that supports “Given-When-Then”
• Fitnesse: a table-driven framework that uses a wiki for displaying and editing tests
• Robot Framework: keyword-driven framework that supports text or tables
• Concordion: Java-based framework for expressing expectations in prose
Copyr
ight
© 2
01
0 Q
ual
ity
Tre
e So
ftw
are,
Inc.
Test “Fixture”
Frameworks, Interfaces, and Drivers
Natural Language
Expectations
GUI
Created in collaboration.
Format defined by the
framework
Code written during development in a
programming language determined by the
framework.
Implementation
“Guts”
Public API
Other interfaces
GUI Driver (e.g. SeleniumRC)
Copyr
ight
© 2
01
0 Q
ual
ity
Tre
e So
ftw
are,
Inc.
Characteristics of ATDD-Friendly Frameworks
• Support expressing expectations in a language and format that fits the context
• Support collaboration among the whole team including developers, testers, & the product owner
• Connect expectations to the system under test with a minimum of test code (“fixtures,” “libraries,” “steps”) to leverage expectations as executable requirements
• Play nicely with source control systems and continuous integration
• Pluggable to support a variety of interfaces
Copyr
ight
© 2
01
0 Q
ual
ity
Tre
e So
ftw
are,
Inc.
Contrasting View: Traditional Test Automation
Written or recorded after the fact.
Expectations are translated, not
leveraged.
GUI
Implementation
“Guts”
Public API
Other interfaces
Automated Test Scripts:
Combination of business-facing expectations and implementation
details.
Back to the Example…
Copyr
ight
© 2
01
0 Q
ual
ity
Tre
e So
ftw
are,
Inc.
Take the Acceptance Tests…
Password Valid? “p@ssw0rd” Yes “p@s5” No “passw0rd” No “p@ssword” No “@#$%1234” No
Given a user is creating an account
When they specify an insecure password
Then they see a message, “Passwords must be at least 6 characters long with at least one letter, one number, and one symbol.”
(Note that these expectations are
implementation-agnostic and express just the
essence of the expectation.)
Copyr
ight
© 2
01
0 Q
ual
ity
Tre
e So
ftw
are,
Inc.
…and Write the Code
Password Valid? “p@ssw0rd” Yes “p@s5” No “passw0rd” No “p@ssword” No “@#$%1234” No
Test “Fixture”
GUI
Implementation
“Guts”
Public API
Other interfaces
Why ATDD? Reason #2: Make progress visible.
Copyr
ight
© 2
01
0 Q
ual
ity
Tre
e So
ftw
are,
Inc.
Are We There Yet?
Why ATDD? Reason #3: Leverage, Efficiency, and Executable Specifications
Copyr
ight
© 2
01
0 Q
ual
ity
Tre
e So
ftw
are,
Inc.
Traditional Approaches
Requirements Management
System
Test Management
System Technical
Specifications
Traceability Matrix
?
Copyr
ight
© 2
01
0 Q
ual
ity
Tre
e So
ftw
are,
Inc.
Test “Fixture”
Efficiency, Reusability, Maintainability
Natural Language
Expectations
The tests define the
requirements.
New interface? Just add a test
fixture
Implementation
(No reconciling multiple, duplicate
artifacts.)
…
New Interface
Implementation change? Make a
localized update to the test fixture.
(Preserve valid expectations.)
(Leverage relevant expectations.)
How does ATDD fit with the rest of the process?
Copyr
ight
© 2
01
0 Q
ual
ity
Tre
e So
ftw
are,
Inc.
ATDD: Part of an Agile Testing Strategy
(A variation on Brian Marick’s Agile Testing Quadrants as published in his essay “Agile Testing Directions”)
Business-facing
Code- facing
Support Critique
Acceptance Tests Exploratory Testing
Unit Tests Reviews, inspections,
pairing, code quality metrics
Copyr
ight
© 2
01
0 Q
ual
ity
Tre
e So
ftw
are,
Inc.
Tests are Versioned with the Code
Natural Language
Expectations Test
“Fixture” Code
Unit Tests
Production Code
Source Control
Copyr
ight
© 2
01
0 Q
ual
ity
Tre
e So
ftw
are,
Inc.
Tests Execute as Part of the Automated Build
Continuous Integration
Images courtesy Mike Clark, www.pragmaticautomation.com. Used with permission.
Why ATDD? Reason #4: No more bugs. (No, I’m not kidding. But yes, there is a catch.)
Copyr
ight
© 2
01
0 Q
ual
ity
Tre
e So
ftw
are,
Inc.
Zero Tolerance for Bugs
!"#$%"%&'%()'*+,
!"#$%"%&'%()'*+,
Copyr
ight
© 2
01
0 Q
ual
ity
Tre
e So
ftw
are,
Inc.
But Not Everything is a Bug
In this context, a BUG
is behavior that violates the letter
or spirit of the Product Owner’s
expectations for the implemented story.
If the behavior does not violate expectations related to the implemented stories, it’s an item
for the backlog.
Given all that… Why not ATDD?
Copyr
ight
© 2
01
0 Q
ual
ity
Tre
e So
ftw
are,
Inc.
Resources
Adzic, Gojko (2009). Bridging the Communication Gap: Specification by Example and Agile Acceptance Testing. Neuri Limited.
Crispin, Lisa (2005). “Using Customer Tests to Drive Development.” Methods & Tools. Summer 2005 Issue. Available online at http://www.methodsandtools.com/archive/archive.php?id=23
Crispin, L., & Gregory, J. (2009). Agile Testing: A Practical Guide for Testers and Agile Teams (Addison-Wesley Signature Series (Cohn)). New York: Addison-wesley Professional.
Marick, Brian (2003). “Agile Testing Directions.” (An explanation of business-facing v. code- facing tests.) Available online at http://www.exampler.com/old-blog/2003/08/22/#agile- testing-project-2