Upload
joemcmahon
View
357
Download
1
Embed Size (px)
Citation preview
C U C U M B E R A N D P E R LJ O E M C M A H O N < J O E . M C M A H O N @ W H I T E H A T S E C . C O M >
– T H E D E V E L O P E R
“These programs run my program and verify it operates correctly.”
AAAAAAAAGHH
– N O N - D E V E L O P E R S
“Just tell me what it does!”
Feature: Provider dashboard As a Provider I want to see my account overview on the dashboard So that when I login I can see information that is important to me without digging for it
Scenario: Provider dashboard Given a provider "Hyper Tiny" belonging to "[email protected]" And an RFP called "Help us!" for "Hyper Tiny" And "Hyper Tiny" has a new endorsement from "Tom Rowley" And "Hyper Tiny" has a new endorsement from "Mike Foley" And I am on the homepage When I log in as "[email protected]" with password "testtest" Then I should see "Help us!" And I should see "Tom Rowley" And I should see "Mike Foley"
B E H AV I O R - D R I V E N T E S T S
• Don’t expose the details of how something happens
• Show me what happens
C U C U M B E R T E S T S
• English-like
• Some leading keywords needed
• Specific way to talk about example data in context
• Mostly free-form
F E AT U R E : W H AT Y O U W A N T
• A short description of the thing you expect to see
• Three lines following are not parsed
• Convention is something like
• As X
• In order to Y
• I want to Z
Feature: Provider dashboard As a Provider I want to see my account overview on the dashboard So that when I login I can see information that is important to me without digging for it
Scenario: Provider dashboard Given a provider "Hyper Tiny" belonging to "[email protected]" And an RFP called "Help us!" for "Hyper Tiny" And "Hyper Tiny" has a new endorsement from "Tom Rowley" And "Hyper Tiny" has a new endorsement from "Mike Foley" And I am on the homepage When I log in as "[email protected]" with password "testtest" Then I should see "Help us!" And I should see "Tom Rowley" And I should see "Mike Foley"
S C E N A R I O : S O M E T H I N G T H AT S H O U L D H A P P E N
• The actual test
• Scenario: describe it
• Given/And: establish conditions
• When: do the thing
• Then/And: what happens
Feature: Provider dashboard As a Provider I want to see my account overview on the dashboard So that when I login I can see information that is important to me without digging for it
Scenario: Provider dashboard Given a provider "Hyper Tiny" belonging to "[email protected]" And an RFP called "Help us!" for "Hyper Tiny" And "Hyper Tiny" has a new endorsement from "Tom Rowley" And "Hyper Tiny" has a new endorsement from "Mike Foley" And I am on the homepage When I log in as "[email protected]" with password "testtest" Then I should see "Help us!" And I should see "Tom Rowley" And I should see "Mike Foley"
S T E P D E F I N I T I O N S
• Each Given/When/Then is backed by a step definition
• The step definition uses a regex to extract data from the step
• Under the hood we’re still writing tests the old way
Given /^a provider "([^\"]*)" belonging to "([^\"]*)"$/
do |provider, email|
provider = Factory.create(
:provider, :company_name => provider)
user = Factory.create(:user, :email => email)
provider.update_attribute(:user, user)
user.update_attribute(:provider, provider)
end
W E L L , Y E A H , B U T T H AT ’ S R U B Y.
T E S T: : B D D : : C U C U M B E R
• Perl implementation of Cucumber (mostly)
• Lets you write Cucumber tests for your Perl code, with step definitions in Perl
L E T ’ S T E S T S O M E T H I N G !
S T E D M A N W H I T W E L L
•A R C H I T E C T •U T O P I A N •W A C K O
R E F O R M I N G P L A C E N A M E S
• He was “troubled” by the recurrence of place names
• Too many Franklins and Springfields
• He proposed a way to name places based on their geographical coordinates
• Rename every place and it’ll be easy to tell where you are.
• …UTOPIA!
W H I T W E L L’ S S C H E M E
S I L LY, B U T S T R A I G H T F O R W A R D
• We transliterate the numbers according to the table, inserting a sign character if needed after the first vowel
• Very much a behavioral thing: I put in this, I get that
Sydney, Australia (33.55S, 151.17E) Fivul Alabee
Moscow, Russia (55.75N, 37.61E) Uleel Feema
Sunnyvale (37.37N, 122.03W) Inin Beveti
Valparaiso, Chile (33.02S, 71.55W) Isite Navul
B U I L D I N G T H E F E AT U R E T E S T
D E S I G N E D T O H E L P Y O U
• Cucumber tries to tell you the next thing you should do
• You don’t have a feature directory, you need one
• You don’t have a feature, make one
• Your feature steps don’t have step definitions, create them
Feature: Test Acme::Geo::Whitwell conversions
As a developer planning to use Acme::Geo::Whitewell::Name
I want to test the conversion of map coordinates to names and vice versa
In order to get a technically correct good laugh
Background:
Given a usable Acme::Geo::Whitwell::Name class
Scenario: Check names match the Whitwell standard
Given coordinates <latitude> and <longitude> for "<real_name>"
When we convert them
Then we get "<name>"
And the first part contains "s" if the latitude is negative or S
And the second part contains "v" if the longitude is negative or W
Examples:
| latitude | longitude | name | real_name |
| 55.75 | 37.61 | Uleel Feema | Moscow |
| -‐33.02 | -‐71.55 | Isite Navul | Valparaiso |
| -‐33.55 | 151.17 | Isilu Buban | Sydney |
| 37.37 | -‐122.03 | Inin Beveti | Sunnyvale |
| 3.8N | 101.42E | Ipou Boubod | Kuala Lumpur |
| 77.85S | 166.76E | Eeseepu Bymeem | McMurdo Base |
# | 82.50 | -‐62.34 | Bleeg Fnord | Alert |
# | 51.51N | 0.13W | A B | London |
# | 7.933S | 14.37E | X Y | Ascension Island |
# Alert is "Eidut Mevik", London is "Ubub Biv", Ascension is "Eesief Bofee".
Feature: Test Acme::Geo::Whitwell conversions
As a developer planning to use Acme::Geo::Whitewell::Name
I want to test the conversion of map coordinates to names and vice versa
In order to get a technically correct good laugh
Background:
Given a usable Acme::Geo::Whitwell::Name class
Scenario: Check names match the Whitwell standard
Given coordinates <latitude> and <longitude> for "<real_name>"
When we convert them
Then we get "<name>"
And the first part contains "s" if the latitude is negative or S
And the second part contains "v" if the longitude is negative or W
Examples:
| latitude | longitude | name | real_name |
| 55.75 | 37.61 | Uleel Feema | Moscow |
| -‐33.02 | -‐71.55 | Isite Navul | Valparaiso |
| -‐33.55 | 151.17 | Isilu Buban | Sydney |
| 37.37 | -‐122.03 | Inin Beveti | Sunnyvale |
| 3.8N | 101.42E | Ipou Boubod | Kuala Lumpur |
| 77.85S | 166.76E | Eeseepu Bymeem | McMurdo Base |
# | 82.50 | -‐62.34 | Bleeg Fnord | Alert |
# | 51.51N | 0.13W | A B | London |
# | 7.933S | 14.37E | X Y | Ascension Island |
# Alert is "Eidut Mevik", London is "Ubub Biv", Ascension is "Eesief Bofee".
Feature: Test Acme::Geo::Whitwell conversions
As a developer planning to use Acme::Geo::Whitewell::Name
I want to test the conversion of map coordinates to names and vice versa
In order to get a technically correct good laugh
Background:
Given a usable Acme::Geo::Whitwell::Name class
Scenario: Check names match the Whitwell standard
Given coordinates <latitude> and <longitude> for "<real_name>"
When we convert them
Then we get "<name>"
And the first part contains "s" if the latitude is negative or S
And the second part contains "v" if the longitude is negative or W
Examples:
| latitude | longitude | name | real_name |
| 55.75 | 37.61 | Uleel Feema | Moscow |
| -‐33.02 | -‐71.55 | Isite Navul | Valparaiso |
| -‐33.55 | 151.17 | Isilu Buban | Sydney |
| 37.37 | -‐122.03 | Inin Beveti | Sunnyvale |
| 3.8N | 101.42E | Ipou Boubod | Kuala Lumpur |
| 77.85S | 166.76E | Eeseepu Bymeem | McMurdo Base |
# | 82.50 | -‐62.34 | Bleeg Fnord | Alert |
# | 51.51N | 0.13W | A B | London |
# | 7.933S | 14.37E | X Y | Ascension Island |
# Alert is "Eidut Mevik", London is "Ubub Biv", Ascension is "Eesief Bofee".
#!perl
use strict;
use warnings;
use Test::More;
use Test::BDD::Cucumber::StepFile;
use Cwd;
Given qr/a usable (\S+) class/, sub { use_ok( $1 ); };
Feature: Test Acme::Geo::Whitwell conversions
As a developer planning to use Acme::Geo::Whitewell::Name
I want to test the conversion of map coordinates to names and vice versa
In order to get a technically correct good laugh
Background:
Given a usable Acme::Geo::Whitwell::Name class
Scenario: Check names match the Whitwell standard
Given coordinates <latitude> and <longitude> for "<real_name>"
When we convert them
Then we get "<name>"
And the first part contains "s" if the latitude is negative or S
And the second part contains "v" if the longitude is negative or W
Examples:
| latitude | longitude | name | real_name |
| 55.75 | 37.61 | Uleel Feema | Moscow |
| -‐33.02 | -‐71.55 | Isite Navul | Valparaiso |
| -‐33.55 | 151.17 | Isilu Buban | Sydney |
| 37.37 | -‐122.03 | Inin Beveti | Sunnyvale |
| 3.8N | 101.42E | Ipou Boubod | Kuala Lumpur |
| 77.85S | 166.76E | Eeseepu Bymeem | McMurdo Base |
# | 82.50 | -‐62.34 | Bleeg Fnord | Alert |
# | 51.51N | 0.13W | A B | London |
# | 7.933S | 14.37E | X Y | Ascension Island |
# Alert is "Eidut Mevik", London is "Ubub Biv", Ascension is "Eesief Bofee".
Feature: Test Acme::Geo::Whitwell conversions
As a developer planning to use Acme::Geo::Whitewell::Name
I want to test the conversion of map coordinates to names and vice versa
In order to get a technically correct good laugh
Background:
Given a usable Acme::Geo::Whitwell::Name class
Scenario: Check names match the Whitwell standard
Given coordinates <latitude> and <longitude> for "<real_name>"
When we convert them
Then we get "<name>"
And the first part contains "s" if the latitude is negative or S
And the second part contains "v" if the longitude is negative or W
Examples:
| latitude | longitude | name | real_name |
| 55.75 | 37.61 | Uleel Feema | Moscow |
| -‐33.02 | -‐71.55 | Isite Navul | Valparaiso |
| -‐33.55 | 151.17 | Isilu Buban | Sydney |
| 37.37 | -‐122.03 | Inin Beveti | Sunnyvale |
| 3.8N | 101.42E | Ipou Boubod | Kuala Lumpur |
| 77.85S | 166.76E | Eeseepu Bymeem | McMurdo Base |
# | 82.50 | -‐62.34 | Bleeg Fnord | Alert |
# | 51.51N | 0.13W | A B | London |
# | 7.933S | 14.37E | X Y | Ascension Island |
# Alert is "Eidut Mevik", London is "Ubub Biv", Ascension is "Eesief Bofee".
Given qr/coordinates (\S+) and (\S+)/, sub {
my $c = shift;
$c-‐>stash-‐>{'scenario'}-‐>{'latitude'} = $1;
$c-‐>stash-‐>{'scenario'}-‐>{'longitude'} = $2;
};
When qr/we convert them/, sub {
my $c = shift;
$c-‐>stash-‐>{'scenario'}-‐>{'name'} =
Acme::Geo::Whitwell::Name::to_whitwell(
$c-‐>stash-‐>{'scenario'}-‐>{'latitude'},
$c-‐>stash-‐>{'scenario'}-‐>{'longitude'}
);
};
Then qr/we get "(.*?)"/, sub {
my $c = shift;
my $name = $1;
is($name, $c-‐>stash-‐>{'scenario'}-‐>{'name'},
'name matches');
};
Feature: Test Acme::Geo::Whitwell conversions
As a developer planning to use Acme::Geo::Whitewell::Name
I want to test the conversion of map coordinates to names and vice versa
In order to get a technically correct good laugh
Background:
Given a usable Acme::Geo::Whitwell::Name class
Scenario: Check names match the Whitwell standard
Given coordinates <latitude> and <longitude> for "<real_name>"
When we convert them
Then we get "<name>"
And the first part contains "s" if the latitude is negative or S
And the second part contains "v" if the longitude is negative or W
Examples:
| latitude | longitude | name | real_name |
| 55.75 | 37.61 | Uleel Feema | Moscow |
| -‐33.02 | -‐71.55 | Isite Navul | Valparaiso |
| -‐33.55 | 151.17 | Isilu Buban | Sydney |
| 37.37 | -‐122.03 | Inin Beveti | Sunnyvale |
| 3.8N | 101.42E | Ipou Boubod | Kuala Lumpur |
| 77.85S | 166.76E | Eeseepu Bymeem | McMurdo Base |
# | 82.50 | -‐62.34 | Bleeg Fnord | Alert |
# | 51.51N | 0.13W | A B | London |
# | 7.933S | 14.37E | X Y | Ascension Island |
# Alert is "Eidut Mevik", London is "Ubub Biv", Ascension is "Eesief Bofee".
Then qr/contains "(\S)" if the (\S+) is negative or (\S)/, sub {
my $c = shift;
my($sign_character, $dimension, $compass_point) = ($1, $2, $3);
my $part = {'latitude' => 0, 'longitude' => 1}-‐>{$dimension};
my $word = (split / /, $c-‐>stash-‐>{'scenario'}-‐>{'name'})[$part];
my $value = $c-‐>stash-‐>{'scenario'}-‐>{$dimension};
my $negative = ($value =~ /^-‐|$compass_point\Z/);
if ($negative) {
ok index($word, $sign_character), 'has negative signifier';
}
else {
is index($word, $sign_character), -‐1, 'no negative signifier';
}
};
D E M O
D E B U G G I N G A S T E P D E F I N I T I O N
• You add the definition, run pherkin, test stays gray
• Either the test didn’t get called (you blew the regex)
• Or the test itself failed to run for some reason
perl -d?
• Way too complex to use directly
• You could eventually find the step definition, but it’s not easy to do by stepping through
• Add $DB::single = 1 to it instead
• Stop is right before the next line to execute
U P S I D E S
• A feature test is way easier to read and understand
• Accessible for non-programmers
• Even writable by non-programmers
• Formalized
• Very free-form
• Easy to work with to get something everyone understands
D O W N S I D E S
• Hidden complexity and “this must be trivial” assumptions
• Still have to write all the test code
• Doesn’t play nice with Test::Unit and the like
• Everybody wants to rule the world
O V E R A L L
• Can help make software make more sense to non-technical folks
• Can help complicated stuff make more sense to technical folks
• More about “what is it supposed to do” than “how does it do this”
T H I S I S A L L V E R Y N I C E …
• …but is anyone using it?
• Yes! WhiteHat uses it for some of our more complex “can X do Y” situations (and we have a lot of X’s)
Feature: Delete a Discussion Response Resource As a Console Operator I want to be able to delete Discussion Responses In order to unclutter the user interface of out of date information
Background: Given a useable NotARealClass::Discussion And a useable NeitherIsThis::Discussion And an Author User And an Operator And a Discussion Resource for a Vuln And a Valid Response
Scenario: Operator tries to delete a discussion response When the Operator calls DELETE on a discussion response Then DELETE response should be successful And the Operator did get one event 'discussion_response_removed' of type 'discussion' Then DELETE response should be forbidden
Scenario: Operator tries to delete a deleted response
Given a Deleted Response
When the Operator calls DELETE on a deleted discussion response
Then DELETE response should be successful
And the Operator did not get one event 'discussion_response_removed' of type 'discussion'
Scenario: Author tries to delete a discussion response
When the Author calls DELETE on a discussion response
Then DELETE response should be forbidden
Scenario: Operator tries delete a non-‐existent response
Given a non-‐existing response_id
When the Operator calls DELETE on a discussion response
Q U E S T I O N S ?
T H A N K S !