Upload
txels
View
330
Download
2
Embed Size (px)
Citation preview
Factories, mocks, spies…
…and other tester’s little helpers
Carles Barrobés twitter: @technomilk github: @txels
Testing is a very broad topic
!
…with its own special lingo
blackbox whitebox regression unit-test
integration-test service-test pyramid
icecream-cone factory assertion spy SuT
…
Let’s start with a question… !
Why do we write tests?
We write tests to save money We tell the computer how to do [tedious] testing for us, faster and cheaper
Writing tests == automation
SuT: System-under-Test*Your “system” as a black box:
I am system !
(with a spec, if you’re lucky)
in
out
in: data, stimuli out: data, observable behaviour
SuT: System-under-TestYour “system” as a white/gray box:
I am system !
(and you can see what’s inside me)
in
out
…BTW I rely on a bunch of external stuff
SuT: System-under-TestYour “system” as a white/gray box:
zin
out
…BTW I rely on a bunch of external stuff
Explicit/Injected dependencies Implicit/Hardcoded
dependencies
Anatomy of [manual] testingTake your code up to the point you want to test Run the specific feature you are testing Verify that it worked
Anatomy of a test casedef test_something_works(): !
prepare <blank line>
exercise <blank line>
verify
aka “Arrange, Act, Assert”
Anatomy of a test casedef test_something_works(): !
prepare <blank line>
exercise <blank line>
verify
Get your code to a known state:
Generate test data Navigate
Isolate and monitor: Set up mocks Set up spies
Anatomy of a test casedef test_something_works(): !
prepare <blank line>
exercise <blank line>
verify
Call your code: result = something(data)
Anatomy of a test casedef test_something_works(): !
prepare <blank line>
exercise <blank line>
verify
Validate results: Assertions on results
Check observed behaviour:
Check reports from your mocks and spies
Time for another question… !
Which of those is the hardest?
Time for the actual talk… !
Let’s look at tools that can help us with the hard
bits
Tools for test setup(I find the preparation phase to be the hardest bit)
[Complex] test data: factories Test objects with behaviour: mocks Instrument internals: spies
FactoriesGoal: make it easy to generate complex test data structures !
Tool of choice: factory boy*
* I’ve tried others, but I prefer this one
Factories: use casesCreate test data with simple statements
Let the factory fill [irrelevant] details
Black/white box testing Explicit dependencies
Factories: simplicityExample: we need a django model instance for our test.
It has lots of mandatory fields… ..but in this test we only care about “title”
Factories: simplicityNot this: publisher = Publisher.objects.create(name=‘Old Books’)
book = Book.objects.create(title=‘Tirant Lo Blanc’,
author=‘Joanot Martorell’,
date=1490,
publisher=publisher)
But this: book = BookFactory(title=‘Tirant Lo Blanc’)
Factory Boy in actionimport factory
from books.models import Book, Publisher
!class PublisherFactory(factory.DjangoModelFactory): FACTORY_FOR = Publisher
name = ‘Test Publisher’
city = ‘Barcelona’
!class BookFactory(factory.DjangoModelFactory): FACTORY_FOR = Book
title = ‘Test Book’
author = ‘Some Random Bloke’
year = 2015
publisher = factory.SubFactory(PublisherFactory)
Maintainability FTW!When you maintain large tests suites, you want to maximise reuse [& DRYness]
Defaults and rules for building your objects live in a central place - easy to adapt
E.g. adding a mandatory field is no longer a pain
Not tied to your test framework
Factory Boy: nicetiesUse a sequence for unique values: name = factory.Sequence( lambda num: 'Name {}’.format(num) ) !
Lazy attributes to populate “late”: slug = factory.LazyAttribute(
lambda obj: slugify(obj.name)
)
Factory Boy: nicetiesFuzzy (randomised) values title = factory.fuzzy.FuzzyText() # u'phPEZzNqfkXv'
gender = factory.fuzzy.FuzzyChoice(('m', 'f')) # 'm'
age = factory.fuzzy.FuzzyChoice(18, 45) # 27 ... !
Coming soon: Faker support (realistic values) name = factory.Faker('name') # u'Isla Erdman'
email = factory.Faker('email') # u'[email protected]'
SpiesGoal: check if something happened inside your code !
Tool of choice: kgb** I don’t know others in Python, used Jasmine (JS)
Spies: use casesWhen it’s hard to have externally observable behaviour
It’s a bit like adding monitoring to your tests
“Blackbox” testing (with some inside knowledge) Implicit dependencies
Spies: how toYou know a little what your system does under the hood You “spy” on a method that should be called (the spy is a wrapper that “calls through”)
Your spy reports on how that method was called
KGB in actionfrom unittest import TestCase
from kgb import spy_on !def add_three(number): return number + 3 !def do_stuff(number): return add_three(number + 1) !class SpyOnTest(TestCase): def test_spy_on_add_three(self):
with spy_on(add_three) as spy: result = do_stuff(15)
self.assertEqual(spy.last_call.args, (16,)) self.assertTrue(spy.called_with(16))
KGB extrasYou can replace the spied on method and make it do nothing or something else
!
with spy_on(SomeClass.add_stuff, call_fake=add_two):
MocksGoal: make it easy to simulate behaviour of a dependency !
Tool of choice: mock*
* There are others, I haven’t tried them
Mocks: use casesYour SuT has explicit callback dependencies (objects it calls)
You want to feed valid objects and inspect what your system did to them
Simulate hard to reproduce conditions (e.g. exceptions)
Mocks vs FactoriesFactories generate “real production objects” Mocks generate fake objects (that you can throw anything at)
mock in action>>> from mock import Mock
>>> user = Mock(username='Fred')
>>> user.username
'Fred'
>>> user.save(force=True)
<Mock name='mock.save()' id='4492633872'>
>>> args, kwargs = user.save.call_args_list[0]
>>> kwargs
{'force': True}
mocking calls>>> user.save.return_value = True
>>> user.save()
True
>>> user.save.side_effect = Exception('Boom')
>>> user.save()
-------------------------------------------------
Exception Traceback (most recent call last)
...
!
Exception: Boom
mock extras: “patch”Patch existing code and replace it with a mock for the duration of a test Similar use cases to spies [but without “call through”]
Mock: “patch” use casesYour SuT has hardcoded dependencies but you want to test it in isolation You want to accelerate your tests [by bypassing expensive calls]
patch in actionfrom mock import patch
!
class MockPatchTest(TestCase):
@patch('test_sample.add_stuff')
def test_do_stuff_calls_add(self, add_stuff):
add_stuff.return_value = ‘whatever'
!
result = do_stuff(123)
!
add_stuff.assert_called_once_with(123)
self.assertEqual(result, 'whatever')
Tools for test validationBuilt-in assert_ functions from your test tool (nose, unittest)
assert statement (if you use py.test it will give useful reporting) Matchers (hamcrest)
MatchersGoal: reusable conditions for assertions !
Tool of choice: hamcrest
Matchers: use casesYou want to check complex or custom conditions in a DRY way
Matchers can be composed - no need for “combinatory” assertions or assertTrue(<complex expression>)
hamcrest highlightsA single assertion: assert_that Many matchers out of the box (plus you can write your own)
Useful reporting on mismatches (no more “False is not True” errors)
Composite matchers: all_of, any_of, not_
hamcrest in actiondef test_any_of(self):
result = random.choice(range(6))
assert_that(result, any_of(1, 2, 3, 4, 5))
!
!
!
AssertionError:
Expected: (<1> or <2> or <3> or <4> or <5>)
but: was <0>
hamcrest in actiondef test_complex_matcher(self):
user = UserFactory()
assert_that(
user.email,
all_of(
not_none(),
string_contains_in_order('@', '.'),
not_(contains_string('u'))
)
)
!AssertionError:
Expected: (not None and a string containing '@', '.' in order and not a string containing 'u')
but: not a string containing 'u' was '[email protected]'
Custom matchersYou can write your own matchers The syntax is a bit verbose, so I wrote matchmaker to make it easier
Custom matchers…from hamcrest.core.base_matcher import BaseMatcher
!
class IsEven(BaseMatcher):
def _matches(self, item):
return item % 2 == 0
!
def describe_to(self, description):
description.append_text('An even number')
!
def is_even():
return IsEven()
…using matchmakerfrom matchmaker import matcher
!
@matcher
def is_even(item):
"An even number"
return item % 2 == 0
Custom matchers in usedef test_custom_matcher(self):
user = UserFactory()
assert_that(user.age, is_even())
!
AssertionError:
Expected: An even number
but: was <19>
More custom matchers@matcher
def ends_like(item, data, length):
"String whose last {1} chars match those for '{0}'"
return item.endswith(data[-length:])
!def test_custom_matcher(self):
user1, user2 = UserFactory(), UserFactory()
assert_that(
user.email,
ends_like(user2.email, 4),
)
!AssertionError:
Expected: String whose last 4 chars match those for '[email protected]'
but: was '[email protected]'
Thanks!Any questions?
Carles Barrobés twitter: @technomilk github: @txels