Unit testing - A&BP CC

Unit Testing the right wayExpressive, useful, and maintainable testing

The Red Line

Goal of Testing

What to Test

Fixtures

Mocks

Assertions

The Red Line

Goal of Testing

What to Test

Fixtures

Mocks

Assertions

GOALS OF TESTING

100% Code Coverage

Test coverage is a useful tool for finding untested parts of a codebase. Test coverage is of little use as a numeric statement of how good your tests are.-- Martin Fowler

public static String foo(boolean someCondition){ String bar = null; if (someCondition) { bar = "blabla"; } return bar.trim();}

100% Code Coverage

assertEquals("blabla", foo(true));

assertEquals("blabla", foo(false));

Line Coverage 100%

Bug Coverage 0%

Reduce Costs

Most of our time is spent on debugging and fixing bugs.Bugs - if not fixed soon in the development process - cost a lot more than

the development itself

Google est. bug cost when fixed at:Development time $5Automated build time $50Integration testing time $500System testing time $5000

Tests as documentation

Tests have to be…

1. comprehensive

2. run often and work

3. written to be read

Tests as documentation

Method names are important to keep tests readableWhile a very subjective topic, a good practice can be:

dividesAmountByFactorthrows_Illegal_Argument_Exception_When_Dividing_By_Zerothrows_exception_when_dividing_by_zero

private double amount;

public double divide(double factor){ if (factor == 0) throw new IllegalArgumentException(); return amount/factor;}

Tests as documentation

● Arrange● Act● Assert

● Given● When● Then

Tests as safety net for refactoring

Tests make you think about your implementation

▪ Tests often trigger refactors and even redesigns

▪ Tests make us really think about the requirements

▪ Tests make us find gaps in the requirements

Conclusion

Write unit tests to…

… reduce costs / save time… be able to confidently refactor… look at your technical design from a different angle… look at your requirements from a different angle

The Red Line

Goal of Testing

What to Test

Fixtures

Mocks

Assertions

WHAT TO TEST

Test isolated units

Keep level of collaboration the smallest possible in units under test

Test isolated units

Smallest possible

Test isolated units

Test in complete isolation

Test isolated units

No knowledge of implementation details of called method of collaborator

Test isolated units

How ?Extensive use of mocked-out collaborators in non-leaf objects/classes

Test isolated units

It results mostly in one test-class per classe.g: MyClass and MyClassTest

Test isolated units

Advantage:Less tests to writeSmaller tests to writeLess complicated setup of fixturesProbably all branches covered quickly

Boundary Cases

Test cases that cover the full spectrum of possible values.

Example 1:A method that accepts only int values between 0 and 100

public void compute(int value){}

Test with -1, 0, 10, 50, 100, 101, 'A'

Boundary Cases

Example 2:A method that removes a character in a String text

public String remove(String text, char ch)

1. Test for null text2. Test for empty text3. Test for character which is not in String4. Test for characters which comes during start, end or middle of String5. Test to cover if text String contains just one character which is equal or not equal to the

to be removed one6. Test with String contains just one character multiple times

Branches

Every time that a different path of execution can be triggered results in a different branch

Basically: if, else, for, while, do blocks and even collaborators

if(condition){

}2 branches : if true and if false

Branches: Cyclomatic complexity

Mathematical result of a formula to calculate the complexity of a piece of code.

Branches: Cyclomatic complexity

Start with a count of one for the method. Add one for each of the following flow-related elements that are found in the method.

Methods Each return that isn't the last statement of a method

Selection if, else, case, default

Loops for, while, do-while, break, and continue

Operators &&, ||, ?, and :

Exceptions catch, finally, throw, or throws clause

Threads start() call on a thread. Of course, this is a ridiculous underestimate!

Branches: Cyclomatic complexity

If this number is higher than 10 it becomes nearly impossible to test.

“Impossible to test” = impossible to have full branch coverage

Branches: Cyclomatic complexity

Keep this number low !!How ?

CLEAN CODE!!!!

Branches: Cyclomatic complexity

Where?SonarQube calculates this out of the box

Code Coverage

Expresses the amount of production code that is covered by automated tests

Code Coverage

Code Coverage: Tools

JaCoCo: replacement for Emma, fully supports Java 7 and 8, used by Sonar, EclEmma(used to be based on EMMA), Jenkins, Netbeans, IntelliJ IDEA, Gradle

Clover: Atlassian → commercial

Cobertura

Conclusion

What to test :▪ Completely in isolation▪ All Boundary cases▪ All branches

Be aware of :▪ Cyclomatic complexity

▪ SonarQube calculates it out of the box for you▪ Branch coverage

▪ Calculation possible by ▪ Jacoco, Emma, Clover, Cobertura

The Red Line

Goal of Testing

What to Test

Fixtures

Mocks

Assertions

FIXTURES

Definition (non-software)

A Fixture in non-software development is e.g.: the setup of a controlled environment to test the functioning of a piece of hardware

Definition (software)

A fixed state of software under test used as baseline for running tests = test context

Preparation of input data and setup/creation of fake or mock objects

Definition (software)

Main benefit:It avoids duplication of code necessary to initialize and clean up common

objects

Use in JUnit

From Junit 4 on there are Java 5 annotations for setting up Test fixture:@Before, @BeforeClass, @After, @AfterClass

@Before, @BeforeClass, @After, @AfterClass

@BeforeClass

public static void prepareSomeResources()

@Before

public void setUpFixtureForAllTests()

@After

public void freeResourcesAfterEachTest()

@AfterClass

public static void freeResourcesAfterAllTestsRan()

ObjectMother

ObjectMother

Factory for creating fixtures that are used in several test classes.

Catchy name thought of at thoughtworks

http://martinfowler.com/bliki/ObjectMother.html

ObjectMother

private PsychoKiller hanibal;

@Before

public void setupFixtureForHanibalLastKill2DaysAgo(){

hanibal = new PsychoKiller();

hanibal.setFirstName(“Hanibal”);

hanibal.setLastName(“Lecter”);

hanibal.setLastKillDate(current - 2 days);

}

ObjectMother

private PsychoKiller hanibal;

@Before

public void setupFixtureForHanibalLastKill2DaysAgo(){

hanibal = PsychoKillerMother.getHanibalLastKill2DaysAgo();

}

@Test

public void returns_true_if_less_than_4_days(){

assertTrue(hanibal.killedRecently());

}

Conclusion

▪ Use Fixtures to reduce code duplication

▪ In JUnit 4 : @Before @BeforeClass @After @AfterClass

▪ ObjectMother: to help keeping setup of fixtures small and concise and

in one place so reusable for other test-classes

The Red Line

Goal of Testing

What to Test

Fixtures

Mocks

Assertions

MOCKS

Why?

Mocking, Spies, Stubs, Doubles & fancy buzzwords

Isolate the code under test

Test one object, not the collaborators !

Speed up test execution

Canned replies are fast !

Make execution deterministic

Less variables means more control !

Simulate special condition

Don’t stick to the happy path.

Gain access to hidden information

Can you see The Hidden Tiger?

Our example codebase

Fancy cars !

Codebase: Car

public class Car {

public void setEngine(final Engine engine) {...}

public void start() {...}

public void stop() { … }

}

Codebase: Engine

public interface Engine {

void start();

void stop();

boolean isRunning();

}

public class DieselEngine implements Engine;

GITHUB!

https://github.com/pijalu/mocktype

Play along !

Dummy|Fake|Stubs|Spies|Mocks

Dummy

Dummy objects are passed around but never actually used. Usually they are just used to fill parameter lists

POJO Dummy

public class TestMockDummy {

final Engine dummyEngine = new DieselEngine();

Car testedCar = new Car(new DieselEngine());

@Test(expected = IllegalStateException.class)

public void testExceptionIfCarStarted() {

testedCar.start();

testedCar.setEngine(dummyEngine);

}

}

Fake

Fake objects have working implementations, but usually take some shortcut which makes them not suitable for production

POJO Fake

private final Engine fakeEngine = new Engine() { boolean started=false;

@Override

public void stop() { started=false; }

@Override

public void start() { started=true; }

@Override

public boolean isRunning() { return started; }

};

POJO Fake (cont)

…

private final Car carWithStartableEngine = new Car(fakeEngine);

@Test

public void testCarCanStart() {

carWithStartableEngine.start();

Assert.assertTrue(carWithStartableEngine.isStarted);

}

…

Stubs

Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what's programmed for the test.

POJO Stubs

private final Engine stubEngine = new Engine() { @Override

public void stop() {}

@Override

public void start() {}

@Override

public boolean isRunning() { return false; }

};

POJO Subs (cont)

private final Car car = new Car(stubEngine);

@Test

public void testCarDoesNotStartWithoutEngine() {

car.start();

Assert.assertFalse(car.isStarted);

}

Spies

NSA your calls.

POJO Spies

private class SpyEngine extends DieselEngine {

int startedCalledCount = 0;

@Override

public void start() {

startedCalledCount++;

super.start();

}

};

POJO Spies (cont)

private final SpyEngine spyEngine = new SpyEngine();

private final Car spiedTestCar = new Car(spyEngine);

@Test

public void testCarStartStartsTheEngine() {

spiedTestCar.start();

Assert.assertEquals("Engine start is called once",

1, spyEngine.startedCalledCount);

}

Mocks

Objects pre-programmed with expectations which form a specification of the calls they are expected to receive.

EasyMock: Mock (too lazy)

@RunWith(EasyMockRunner.class)

public class TestMockEasyMock

extends EasyMockSupport{

@Mock

Engine mockEngine;

…

EasyMock: Mock (cont)

@Test

public void testCarWithAMockedEngine() {

// Set the behavior

expect(mockEngine.isRunning()).andReturn(false);

mockEngine.start();

expect(mockEngine.isRunning()).andReturn(true);

mockEngine.stop();

replayAll();

EasyMock: Mock (cont 2)

// Set a car with our mocked engine

Car testedCar = new Car(mockEngine);

// Run !

testedCar.start();

testedCar.stop();

// Verify !

verifyAll();

}

POJO feels a bit ghetto ?

(Some) Existing frameworks

▪ EasyMock

▪ Mockito

▪ Unitils

▪ JMockit

And Much Much more !

Proxy Based

Class remap

Strict by default

Non-Strict by default

EasyMock

1. Arrange: Create the mock and setup behaviormock = createMock(Collaborator.class);mock.documentChanged("Document");expectLastCall().times(3);expect(mock.vote("Document")).andReturn((byte)-42);replay(mock);

3. ACT !classUnderTest.addDocument("Document", "content");

4. Assertverify(mock);

Mockito - Behavior Check

1. ArrangeList mockedList = mock(List.class);

2. ActmockedList.add("one");mockedList.clear();

3. Assertverify(mockedList).add("one");verify(mockedList).clear();

Mockito - Stubbing

1. ArrangeLinkedList mockedList = mock(LinkedList.class); when(mockedList.get(0)).thenReturn("first");when(mockedList.get(1))

.thenThrow(new RuntimeException());2. Act (examples)System.out.println(mockedList.get(0));System.out.println(mockedList.get(1));System.out.println(mockedList.get(999));

Unitils (mocking)

1. Arrange (Create the mock and setup behavior)myServiceMock =

new MockObject<MyService>(MyService.class, this);myServiceMock.returns("a value").someMethod();

2. ActmyServiceMock.getMock().someMethod();

3. Assert !myServiceMock.assertNotInvoked().someMethod();

JMockit - Expectations

@Testpublic void aTestMethod(@Mocked final MyCollaborator mock){ new NonStrictExpectations() {{ mock.getData(); result = "my test data"; mock.doSomething(anyInt, "some expected value", anyString);

times=1; }}; // In the replay phase, the tested method would call the "getData" and "doSomething" // methods on a "MyCollaborator" instance. ...

// In the verify phase, we may optionally verify expected invocations to // "MyCollaborator" objects. ...

}

JMockit - Verification

... new Verifications() {{ // If no new instance of the mocked class should have been // created with the no-args constructor, we can verify it: new MyCollaborator(); times = 0; // Here we verify that doSomething() was executed at least once: mock.doSomething(); // Another verification, which must have occurred no more than three // times: mock.someOtherMethod(

anyBoolean, any, withInstanceOf(Xyz.class)); maxTimes = 3; }};}

Conclusion

▪ Mocking helps you to create better tests▪ Mocking Framework(s) help keeping overhead low

The Red Line

Goal of Testing

What to Test

Fixtures

Mocks

Assertions

ASSERTIONS

Assertions

What ? Verify if the result of the test is what we expected.

Assertions

How ? Should be automated, no human intervention necessary No logging, System.out.println() etc. to be used Use framework(s) to achieve this.

Junit assertions

TypesassertEquals, assertSame, assertTrue, assertFalse, assertNull, assertNotNull

Advantages Most often used, so best known.

Junit assertions

Drawbacks▪ Not always very readable

assertEquals(expected, result) or assertEquals(result, expected)

▪ Number of assertions are limited▪ Comparing (complex) objects is hard.

impossible when object has not equals() implemented !

ConclusionIt’s ok to use them for primitive types.But there are better alternatives.

Hamcrest assertions

What ?▪ Hamcrest is a framework for writing matcher objects allowing ‘match’

rules to be defined declaratively.assertThat(Object, Matcher<T>);

▪ Can easily be integrated with other frameworks like Junit, TestNG, Mockito, EasyMock, Jmock,...

Hamcrest assertions

Advantages▪ Improved readability of tests

assertThat(ObjectToBeChecked, equalTo(OtherObject))

assertThat(ObjectToBeChecked, is(equalTo(OtherObject));

assertThat(collection, hasSize(2);

▪ Better failure messageassertThat(3, greaterThan(5));

Expected: a value greater than <5>

but: <3> was less than <5>

Hamcrest assertions

▪ Combination of MatchersAllows to assert more precisely.

assertThat(array, not(emptyArray());

assertThat(collections, everyItem(greaterThan(10));

▪ Write your own Matcher

This can occur when you find a fragment of code that test the same set of properties over and over again and you want to bundle the fragment into a single assertion.

But be aware, there are already plenty of matchers available, make sure you are not writing existing code again.

Hamcrest assertions

Example : To test if a double has a value NaN (not a number)

Test we want to write :

public void testSquareRootOfMinusOneIsNotANumber () {

assertThat(Math.sqrt(-1), is(notANumber()));

}

public class IsNotANumber extends TypeSafeMatcher<Double> {

@Override

public boolean matchesSafely(Double number) {

return number.isNaN();

}

public void describeTo(Description description) {

description.appendText("not a number");

}

@Factory

public static <T> Matcher<Double> notANumber() {

return new IsNotANumber();

}

}

Hamcrest assertions

Hamcrest assertions

▪ More possibilities to compare objects

It is possible to check objects that don’t have equals() implemented.

assertThat(ObjectToBeChecked, samePropertyValuesAs(OtherObject));

but not possible for objects with composition !

Better use ReflectionAssert.assertReflectionEquals of unitils !

Hamcrest assertions

Drawbacks▪ Finding the right Matcher

The matchers are not set in one place. Most matchers are accessible via the Matcher class, but some are located in the CoreMatcher class, and some are in another package.

example:

hasItem() : Matcher classhasItems() : IsCollectionContaining class

Other frameworks

Unitils▪ ReflectionAssert

This assertion loops over all fields in both objects and compares their values using reflection. If a field value itself is also an object, it will recursively be compared field by field using reflection. The same is true for collections, maps and arrays.

▪ Lenient assertionsAdding some levels of leniency to the ReflectionAssert checks. (order list, ignoring defaults, dates, assertLenientEquals)

▪ Property assertionsMethods to compare a specific property of two objects

assertPropertyLenientEquals("id", 1, user);

assertPropertyLenientEquals("address.street", "First street", user);

Other frameworks

Fest (http://code.google.com/p/fest/)

Supports both Junit and TestNG

assertThat(collection).hasSize(6).contains(frodo, sam);

Assertions

Misuse of assertions▪ Manual assertions

This practice misses out the main benefits of testing automation — the ability to continuously run the tests in the background without intervention

■ Multiple assertions

■ Redundant assertionsExtra calls to an assert method where the condition being tested is a hard coded valueassertTrue(“always true”, true)

■ Using the wrong assertionsassertTrue("Object must be null", actual == null);assertTrue("Object must not be null", actual != null);

Assertions

What about void methods ?Often if a method doesn't return a value, it will have some side effect. There may be a way to verify that the side effect actually occurred as expected.

Especially exception testing should not be forgotten.

Assertions

MyClass {

public void addElement(String element, List<String> elements) {

elements.add(element);

}

}

public void testAddElement() {

List<String> elements = new ArrayList();assertEquals(0, elements.size() );myClassTest.addElement(“test”, elements);assertEquals(1, elements.size() );

}

Conclusion

▪ Always make sure your assertions are fully automated

▪ Junit assertions are ok for primitive types

▪ Hamcrest offers a lot of interesting matchers that allows you to assert

more precise

▪ Unitils is a better alternative when comparing objects

▪ You can use them all together !

The Red Line

Goal of Testing

What to Test

Fixtures

Mocks

Assertions

Question ?

PRACTICAL

Exercises !

github: https://github.com/pijalu/utdemo

▪ Master: Default start

▪ Branches: Different solutions with different mocking framework…

Don’t look ;-)

Factoid - Model

Fact

▪ Simple POJO▪ Stores a fact▪ It’s a string

Factoid - Provider

Provider interface:▪ Provides a list of fact to a client

▪ int size(): Number of facts in the provider▪ Fact getFact(index): Return a fact (0->size-1)

▪ Implementation:▪ FileFactProvider

▪ Loads facts from a file.▪ Line oriented

Factoid- Service

FactService interface:▪ Returns a fact to client:

▪ Fact getAFact(): Return a fact

▪ Implementation:▪ RandomFactService:

▪ Returns a random fact using a provider▪ Uses Random▪ Builds an array to avoid repetition/ensure all facts are returned

Factoid - Main

Factoid main class

▪ Loads a File with a (File)FactProvider

▪ Loads a (Random)FactService using created fact provider

▪ Calls FactService getAFact()

Factoid - What to do

▪ Select the mocking framework you want▪ EasyMock to start !

▪ Check the FIXME in the existing code

▪ Fix as many as you can !▪ Be creative

▪ Look for other issues ;-)

GO !

Thanks for your attendance