Testing outside of the Ruby World

T!"#$%& O'#"$(! )f #*!

R'b+ W)r,(Joseph Wilk

Wait isn’t this a Ruby conference?

Seriously...

No Ruby tools.

Really.

Science

“The number of languages you know corresponds to your programming

skill”

JUnit

Rspec

PHPSpec

JSpec

Circumspec

SomethingSpec

Yay... TestingScrewUnit

JBehave

Jasmine

Cucumber

BlahSpec

WhateverSpec

JUnit

Rspec

PHPSpec

JSpec

Circumspec

SomethingSpec

Yay... TestingScrewUnit

JBehave

Jasmine

Cucumber

BlahSpec

WhateverSpec

BORING!

T.ing & Mocking

Java

Ruby

Monkey.stub!(:new).and_return(mock("monkey"))

I owe you one Java mocking example.

I don’t have the will power to write it. Sorry.

T.ing & Mocking

Java

Ruby

Monkey.stub!(:new).and_return(mock("monkey"))

I owe you one Java mocking example.

I don’t have the will power to write it. Sorry.

BORING!

InterestingAsynchronous Property testing

Model testingPermutation explosions

Test feedbackMetrics

Graphical tests

HaskellCurry

“Program testing can be used to show the presence of bugs, but never to show their absence!”

Edsger Dijkstra

QuickCheckProperties

∀s . length(five_random_characters(s)) = 5

For all values of s the length of the thing returned by five_random_characters is 5

QuickCheck

LogicRandomly generate

testsFunctionFunction

Properties

QuickCheck

QuickCheck

LogicRandomly generate

testsFunctionFunction

Properties

QuickCheck

QuickCheck

LogicRandomly generate

testsFunctionFunction

Properties

QuickCheck

QuickCheck

LogicRandomly generate

testsFunction

Properties

QuickCheck

QuickCheck

LogicRandomly generate

testsFunction

Properties

QuickCheck

Counter Examples

QuickCheckProperties

it "should reverse a string" do "monkeys".reverse.reverse.should == "monkeys"end

100.times.map {“#{rand(10)}#{rand(10)}”}.each do |char| it "should reverse a string" do char.reverse.reverse.should == char endend

QuickCheckimport Data.Charimport Test.QuickCheck

instance Arbitrary Char where arbitrary = choose ('\32', '\128') coarbitrary c = variant (ord c `rem` 4)

prop_RevRev xs = reverse (reverse xs) == xs where types = xs::[Char]

Properties

$ Main> quickCheck prop_RevRevOK, passed 100 tests.

ErlangMessaging/

Concurrency

Erlang runtimesystem

McErlang runtimesystem

McErlangModels

communication

concurrency

distribution

McErlangModels

MessengerService

Message client

Message client

Fred Clara

message“Scottish fiction”

“Scottish fiction”

login login

McErlangif user1 does not send a message m to user2 until user2 is logged on, then if user1 does send a message m to user2 then eventually user2 receives the message m.

"not P until Q => (eventually P => eventually R)”

P: clara sends message “Scottish fiction” to fredQ: fred is logged onR: fred receives the message “Scottish fiction” from clara

Models

McErlangModels

{program={scenario,start,[[[{logon,clara},{message,fred,"hi"},logoff],[{logon,fred},logoff]]]},

monitor={mce_ltl_parse:ltl_string2module_and_load("not P until Q implies (eventually P implies eventually R)", messenger_mon),{void,[{'P',basicPredicates:message_to (clara,fred,"hi")}, {'Q',basicPredicates:logon(fred)}, {'R',basicPredicates:message_received(fred,clara,"hi")}]}}, algorithm={mce_alg_buechi,void}}).

“Every method you use to prevent or find bugs leaves a residue of subtler bugs against which those methods are ineffectual

Pesticide Paradox / Beizer

ClojureBracket

hell

(fact (alive-in-next-generation? ...cell...) => truthy   (provided     (alive? ...cell...) => false    (neighbor-count ...cell...) => 3))

MidjeFacts

cell = mock("a cell")cell.stub(:alive?).and_return(false)cell.stub(:neighbour_count).and_return(3)

cell.alive_in_next_generation.should == true

IokeBrief visit

IokeSpecs are documentation

IokeSpecs are documentation

JavaScriptWithout the Java

Zombie.jsTrapped inside a browser

Zombie.jsTrapped inside a browser

var zombie = require("zombie");var assert = require("assert");

zombie.visit("http://localhost:3000/", function (err, browser, status) {

browser. fill("email", "zombie@underworld.dead"). pressButton("Sign Me Up!", function(err, browser, status) {

assert.equal(browser.text("title"), "Welcome"); })});

VowsTopics

{ topic: function () { return 42 },

'should be a number': function (topic) { assert.isNumber (topic); }, 'should be equal to 42': function (topic) { assert.equal (topic, 42); }}

VowsAsynchronous calls

{ topic: function () { fs.stat('~/FILE', this.callback); }, 'can be accessed': function (err, stat) { assert.isNull (err); // We have no error assert.isObject (stat); // We have a stat object }, 'is not empty': function (err, stat) { assert.isNotZero (stat.size); // The file size is > 0 }}

VowsPromises / Futures

{ topic: function () { var promise = new(events.EventEmitter); fs.stat('~/FILE', function (e, res) { if (e) { promise.emit('error', e) } else { promise.emit('success', res) } }); return promise; }, 'can be accessed': function (err, stat) { assert.isNull (err); //We have no error assert.isObject (stat); //We have a stat object }, 'is not empty': function (err, stat) { assert.isNotZero (stat.size); //The file size is > 0 }}

VowsParallel Execution

{ '/dev/stdout': { topic: function () { path.exists('/dev/stdout',this.callback) }, 'exists': function (result) { assert.isTrue(result) } }, '/dev/tty': { topic: function () { path.exists('/dev/tty',this.callback) }, 'exists': function (result) { assert.isTrue(result) } }, '/dev/null': { topic: function () { path.exists('/dev/null',this.callback) }, 'exists': function (result) { assert.isTrue(result) } }}

Permutation ExplosionTestSwarm

Permutation ExplosionTestSwarm

JavaReally

JunitM3Faster Test feedback

Lots of very short testsA few very long ones

Failures are not randomly distributed

Kent Beck

JunitM3Faster Test feedback

Industrial LogicLearn from Metrics

Other stuffdessert

‘‘What is the use of a book,’’ thought Alice, ‘‘without pictures or conversations?’’

Lewis Carroll Alice’s Adventures in Wonderland

Swim LaneWords are not enough

Ward Cunningham

Ward Cunningham

http://vimeo.com/22165070

Graphical Testing

Brian Marick

http://testobsessed.com/wp-content/uploads/2007/02/testheuristicscheatsheetv1.pdf

‘‘How much do you know about the heuristics of failure?’

Joseph WilkScotland Ruby Conf 2011

Tasty Ideas

5anks!Joseph Wilk

@josephwilkhttp://blog.josephwilk.net