Successful Unit testing for large applications

SUCCESSFUL UNIT

TESTING FOR LARGE

APPLICATIONS

Erik LeBel

elebel@pyxis-tech.com

© P

yxis

Te

chnolo

gie

s in

c.

PYXIS STUDIO

SETTING THE SCENE

© P

yxis

Te

chnolo

gie

s in

c.

You are trying to reduce frequency with

which regressions get pushed into

production

You are trying to speed up validation for

quicker release cycles

You are cool

You‟ve been told to do so by someone

higher up in your organization

IF YOU ARE USING AUTOMATED

TESTING, THEN…

© P

yxis

Te

chnolo

gie

s in

c.

Your project has tests, but they aren't

helping you deliver stable software

Your project has tests, but they‟re a

nightmare to maintain

Your project has tests, but they are a

haphazard mess in the making

You like tests

YOU ARE INTERESTED IN THIS

PRESENTATION BECAUSE…

© P

yxis

Te

chnolo

gie

s in

c.

We will be covering:

practices and concerns to help make

automated tests an enabling force

We are not discussing:

why we automate testing

© P

yxis

Te

chnolo

gie

s in

c.

SO, WHAT‟S THIS ABOUT DEBILITATING

TESTS?

© P

yxis

Te

chnolo

gie

s in

c.

Tests:

Are hard to understand

Break frequently

Are slow to execute

Are hard to fix

Let bugs through

Don‟t validate anything

useful

But we want them:

To be clear to

understand

To break legitimately

To be fast to execute

To be concise

THIS IS OFTEN BECAUSE…

MAKE TESTS

READABLE

© P

yxis

Te

chnolo

gie

s in

c.

Use OO practices to:

Make setup code easy to read

Unclutter

Use factory methods

Use fluent builders

Can build complex actions as well as

objects

CLEAN UP TEST SETUP

© P

yxis

Te

chnolo

gie

s in

c.

Simplify result validation

Can easily handle unordered cases

Write your own if your test framework

doesn‟t have any

COLLECTION ASSERTS

© P

yxis

Te

chnolo

gie

s in

c.

They are easier to read

USE LITERALS

© P

yxis

Te

chnolo

gie

s in

c.

Use extension methods for concise type

assertions

Build utility methods for complex

validations

BUILD INTUITIVE ASSERTIONS

© P

yxis

Te

chnolo

gie

s in

c.

Domain Specific Languages (DSLS)

Test API wrappers

Model wrappers

USE TEST DSLS FOR INTEGRATION

TESTS

© P

yxis

Te

chnolo

gie

s in

c.

Capture large expected value sets in

common file formats such as XML, JSON,

YAML

Write assertion utilities that highlight

multiple differences between expected and

actual values

(eg XmlUnit)

USE READABLE DATA FORMATS

© P

yxis

Te

chnolo

gie

s in

c.

You will more easily spot redundant or

useless tests

Tests should start to match business

requirements

Its should start to be clearer where there are

gaps

The factories, builders and helpers that you

develop for testing will make it easier to add

new tests scenarios

AS TESTS GET CLEANER

© P

yxis

Te

chnolo

gie

s in

c.

„If‟ statements in tests

Loops in tests

Exception handling

AT ALL COST, AVOID…

BALANCE

© P

yxis

Te

chnolo

gie

s in

c.

THE NUMBERS SHOW THAT…

System tests

Integration tests

Unit tests

Time to execute

Fragility

Lines of test code

Effort to fix

Overall cost of use

$$$

$

© P

yxis

Te

chnolo

gie

s in

c.

Module1.dll

Module1.Unit.Tests.dll

Module1.Integration.Tests.dll

Module2.dll

Module2.Unit.Tests.dll

Module2.Integration.Tests.dll

BUILD YOUR TESTS AROUND YOUR

MODULES

© P

yxis

Te

chnolo

gie

s in

c.

Configurations

Databases

File systems

External services

Sandboxing technologies

Flashy, shiny widgets

Use them… don‟t let it be the other way

around

TECHNOLOGIES THAT HINDER TESTING

© P

yxis

Te

chnolo

gie

s in

c.

Don‟t expect to stop delivering software while fixing things

Don‟t be afraid to experiment

When a pattern emerges, make sure it‟s communicated to the whole team

Don‟t get hung up on the “green field” vs “brown field” over time even the greenest fields get muddy

This is and will continue to be an organic process

GETTING THERE IN BABY STEPS

BUILDING TRUST

© P

yxis

Te

chnolo

gie

s in

c.

Commit to keeping tests green all the time.

If it‟s broken, fix it, or delete it.

DO OR DO NOT

© P

yxis

Te

chnolo

gie

s in

c.

Use a Continuous Integration tool:

TFS, CruiseControl.NET, Hudson/Jenkins,

roll your own

Don‟t allow the build to stay broken

Make sure your integration build and tests

are run as often as possible

Split long-running test suites to run as part

of an alternate set of build jobs

AUTOMATE TEST EXECUTION

© P

yxis

Te

chnolo

gie

s in

c.

Work to improve test coverage

Beware of high test coverage and low

software quality

And remember it‟s just an indicator

WATCH CODE COVERAGE

© P

yxis

Te

chnolo

gie

s in

c.

Monitor test and production failures

Use production issues to infer how deep

the iceberg is

When something fails, find the cause, and

improve your test and application design

INSPECT AND ADAPT

© P

yxis

Te

chnolo

gie

s in

c.

Test the reliability of your tests by seeding

bugs and checking the detection rate

Automate the activity with a mutation

testing framework:

See http://www.mutation-testing.net

FAULT SEEDING

CONCLUSION

© P

yxis

Te

chnolo

gie

s in

c.

Treat your tests as first order citizens

Work to improve your feedback cycle

Commit or get out, but don‟t go in halfway

TAKE AWAY

Titre sur mesure

POINTS FORTS

1

Thank you!

Questions?

pyxis-tech.com

Erik LeBel

elebel@pyxis-tech.com