Test Driven development

Tor Stålhane

What we will cover

We will cover three aspects of testing

• Testing for green-field projects. This is TDD as it was originally understood.

• Testing for changing legacy code

• TDD and acceptance testing

Development and testing

In TDD, it is unreasonable to completely separate testing and implementation.

When we decide on a implementation strategy, we have also indirectly chosen a test strategy.

The reason for this is that we have to build tests for what we later will implement.

Why TDD

Important advantages of TDD are that it encourages:

• Writing clear requirements

• Development in small steps. This will make debugging easier since we will have small code chunks to debug.

• Minimalistic code and enforce the YAGNI principle – “You Ain’t Gonna Need It”

Green field projects

Where to start

TDD operates with four pairs of strategies which encompass testing and code. We will look at each pair in some detail.

• Details vs. the “big picture”

• Uncertain territory vs. the familiar

• Highest values vs. the low-hanging fruits

• Happy paths vs. error situations.

Details vs. the “big picture”

• Start with the details:– Solve all the small problems before we

combine them into a component

• Start with the “big picture”:– Solve the design problems – e.g. component

structure and them include all the details

In most cases, we will start with writing tests and then code for our greatest concerns.

Uncertain territory vs. the familiar

The question here is about priorities:• Exploring uncertain territory – reduce risk.• The familiar – what is best understood –

will in many cases bring larger user benefits faster.

We an thus look at this as a cost/benefit problem – reduced risk vs. immediate benefits. We will write test and then code in away that give us the best cost/benefit ratio

Highest value vs. the “low-hanging fruits”

This is a variation of the previous topic – familiar vs. unknown – and is thus also a cost/benefit question.

Going for the “low-hanging fruits”, we can demonstrate a lot of functionality early in the project without spending too many resources.

Since it is easier to write both tests and code for the “low-hanging fruits” this is the popular alternative

Happy paths vs. error situations.

The choice between a happy path and an error situation is mostly an easy choice.

Even if the robustness requirements are extreme, we will first need tests and code that do something useful for the user.

There are exceptions – situations where we will need the error handling quite early. A typical example is a log-on function where we need error handling like “unknown user” and wrong password right from the start.

Essential TDD concepts

We will walk through the following concepts:

• Fixtures

• Test doubles

• Guidelines for a testable design

• Unit test patterns

• Legacy code

Fixtures

A fixture is a set of objects that we have instantiated for our tests to use.

Test Doubles - 1A test double is an object “stand-in”. Typical

features are that it:• Looks like the real thing from the outside• Execute faster• Is easier to develop and maintain

Test doubles are used for two types of testing:

• State based testing• Interaction based testing

Test Doubles - 2

We can sum state based and interaction based testing in that we use

• Interaction based testing to verify how an object talks to its collaborators. Am I using the objects around me correctly?

• State based testing how well the object listens. Am I responding correctly to the input and responses that I get from others?

Test Doubles – 3 We have three types of test doubles:• Stubs: the simplest possible

implementation of an interface• Fakes: more sophisticated than a “stub” –

e.g. an alternative, simpler implementation• Mocks: more sophisticated than a “fake”. It

can contain– Assertions– The ability to return hard coded values– A fake implementation of the logic

Why use test doubles – 1

Important reasons:• Can test an object without writing all its

environment• Allows a stepwise implementation as we

successively replace more and more doubles

• When something is wrong we can be almost sure that the problem is in the new object and not in its environment.

Why use test doubles – 2

Real 1

Double 1

Double 2

Double 3

Real 1

Real 3

Real 4

Real 2

21

Real 1

Real 3

Double 2

Real 2 3

Real 1

Double 1

Double 2

Real 2

4

Unit-testing patterns – 1 The most important unit test patters are

assertion patters. A typical example is shown below.

public static void assertEquals(java.lang.String expected,

java.lang.String actual)

Asserts that two strings are equal. Throws an AssertionFailedError if not.

Unit-testing patterns – 2 Sourceforge uses six basic assertions:• assertFalse. Check that the condition is false• assertTrue. Check that the condition is true• assertEquals. Check that two parameters are

equal.• assertNotEquals. Check that two parameters are

not equal.• assertNotSame. Check that two objects do not

refer to the same object• fail. Fail a test

Unit-testing patterns – 3

In the general case, an assertion should be placed

• As close as possible to the new code – for instance right after the last statement.

• Right after each important new chunk of code.

The intention is to discover any error as soon as possible.

Unit-testing patterns – 4

It is important to discover any error as soon as possible. By doing this we have less code to go through in order to find the cause of the error.

In order to achieve this, we need assertions whenever we want to check that we are on the right course.

Assertion types

An assertion’s type depends on its usage. The most important types are:

• Resulting state assertion

• Guard assertion

• Delta assertion

• Custom assertion

• Interaction assertion

Resulting state assertion

The main idea here is to

1. Execute some functionality

2. Check that the resulting state is as expected. This is done using one or more assertions – most often assertEquals assertions.

Guard assertionThe guard assertion is used to check that our

assumptions of what the status is before executing the new code is correct. E.g.:

assertTrue(java.lang.String message, boolean condition)

New code to be tested

assertNotSame(java.lang.Object expected, java.lang.Object actual)

Delta assertion – 1

A delta assertion is an assertion where we, instead of checking an absolute value, checks the expected delta – change – of the value. This has at least two advantages:

• We do not a “magic number” in the test.

• The test will be more robust to changes in the code.

Delta assertion – 2

test_var_1 = current value

new code which, among other things, increase the current value by x

assertEquals(test_var_1, test_var_1 + x)

This assertion will hold whatever value we have for test_var_1.

Custom assertion – 1 Custom assertion covers a wide range of

customized assertion patterns – also called “fuzzy assertions”.

• Con: they have to be tailored to the assertion that we want to make.

• Pro: – They can cover a wide range of important test

situations. – Over time we can build up a library of custom

assertions thus reducing the need for writing new ones.

Custom assertion – 2

In most cases we can obtain the custom assertion effect by using more complex predicates and expressions in the usual assertions. However, the custom assertion will enable better error handling and allow more informative error messages.

Custom assertion – examplecustom_assertInrange(in_value,

message)

assertTrue(message, (in_value < max) && (in_value > min))

As the conditions get more complex, the custom assertion becomes a more attractive alternative

Interaction assertion – 1

The interaction assertion does not check that our code works as expected.

It role is to check that our code cooperate with our collaborator objects as expected.

This can be done using any of the assertion patterns that we have discussed earlier, e.g. assertFalse, assertSame, assertEquals or assertTrue.

Interaction assertion – 2

A typical case where we would use an interaction assertion is as follows:

• We download a freeware component plus some documentation.

• Based on the documentation we believe that if we do “A” then the component will return “B”.

• Use an assertion to check if this is true.

Keep or throw away – 1

We need to decide whether we want to keep the assertion in the production code or not.

• Pro keeping it:Will quickly diagnose new errors.

• Con keeping it:May need updates when the code is changed, e.g. during maintenance

Keep or throw away – 2

The decision may vary depending on the assertion pattern used.

E.g. the delta pattern and the custom pattern will be more robust against code changes than for instance the resulting state pattern, especially if it contains one or more magic numbers.

Legacy code

Legacy code – 1

Much of the development done – may be even most of the development done – is changing an existing code base.

TDD assumes that we write tests, then write code and then run the tests. This approach is not useful for legacy code and we cannot use TDD directly as described earlier.

Legacy code – 2

The following process should be used:1. Identify the change point in the legacy

code2. Identify an inflection point3. Cover the identified inflection point

– Break internal dependencies– Break external dependencies– Write tests

4. Make changes 5. Refactor covered code

Legacy code – 3

Note that the test in step 3 on the previous slide are not tests to check your changes.

These tests are called characterization tests. Their purpose is to help you understand the behavior of the current code. We could thus apply interaction assertion patterns even if there may be no real interactions

Inflection points – 1 An inflection point – also called a test point

– is defined as follows:

An inflection point is a point downstream from the change point in your code where you can detect any change in the code’s behavior relevant to your changes.

Everything else considered equal, we will prefer an inflection point as close as possible to the change point.

Inflection points – 2

We will use a remote inflection point if we e.g. are con concerned with side effects from our changes. The side effects may show up only late in the execution and thus a distant inflection point will be a better solution.

Test and change

As we change the legacy code we will use both the characterization tests and the new-code tests.

• New code tests to check that our changes have the intended effect

• Characterization tests to check that we o not break any of the behavior that we want to keep.

Acceptance testing

The TDD acceptance testing process

Pick a user story Write tests forthe story

Automate tests Implement functionality

TDD acceptance testing

We will look in more details on the three first steps of the process:

• Pick a user story

• Write tests for the story

• Automate tests

Pick a user story – 1

We have already partly covered this in an earlier section – “Where to start”. Thus, in many cases, the user stories already are prioritized.

Otherwise, the user stories should be prioritized based on:

• Business value – what is it worth to the customer.

• Technical risk – how difficult is it to implement.

Pick a user story – 2

More formally, we can use business value and technical risk to assess the leverage of each user story:

Leverage =

(Business value – Tech. risk) / Tech. risk

Pick a user story – 3

When we reach this stage in system development, developers, testers and customers already have had a lot of informal discussions on what the customer needs, when and why.

This knowledge will be included in the decision of which user story to pick next.

Write tests for the story – 1

One common way to do this is as follows:

• Sit down with the customer and describe the most important scenarios for this user story.

• Reach an agreement on what functionality needs to be implemented in order to realize each scenario.

Write tests for the story – 2

The results of the process described on the previous slide are:

• A set of well defined tests – a testing plan for the user story

• Expected results – acceptance criteria – for each test. This must include both:– Expected output– Expected new system state

Automate the tests

How to automate tests will depend on the tools used. A the present, it seems that table presentation ala FITNESS are much used. In all cases we need:

• Id of function to be tested

• Input parameters

• Expected result

See also the earlier FITNESS presentation

Summing up

The most important thing to remember about TDD is that the concept contains both testing and development.

This does not imply that we must use an agile development process.

However, it implies that we must consider implementation and testing as two parallel activities – we cannot ignore one and only focus on the other.

Acknowledgement

Most of the content of these slides are taken from chapters 4 and 5 inthe book “Test Driven” by L. Koskella.

More on assertions can be found athttp://junit-addons.sourceforge.net/junitx/framework/Assert.html