Unit Testing the right wayExpressive, useful, and maintainable testing
The Red Line
Goal of Testing
What to Test
Fixtures
Mocks
Assertions
The Red Line
Goal of Testing
What to Test
Fixtures
Mocks
Assertions
GOALS OF TESTING
100% Code Coverage
Test coverage is a useful tool for finding untested parts of a codebase. Test coverage is of little use as a numeric statement of how good your tests are.-- Martin Fowler
public static String foo(boolean someCondition){ String bar = null; if (someCondition) { bar = "blabla"; } return bar.trim();}
100% Code Coverage
assertEquals("blabla", foo(true));
assertEquals("blabla", foo(false));
Line Coverage 100%
Bug Coverage 0%
Reduce Costs
Most of our time is spent on debugging and fixing bugs.Bugs - if not fixed soon in the development process - cost a lot more than
the development itself
Google est. bug cost when fixed at:Development time $5Automated build time $50Integration testing time $500System testing time $5000
Tests as documentation
Tests have to be…
1. comprehensive
2. run often and work
3. written to be read
Tests as documentation
Method names are important to keep tests readableWhile a very subjective topic, a good practice can be:
dividesAmountByFactorthrows_Illegal_Argument_Exception_When_Dividing_By_Zerothrows_exception_when_dividing_by_zero
private double amount;
public double divide(double factor){ if (factor == 0) throw new IllegalArgumentException(); return amount/factor;}
Tests as documentation
● Arrange● Act● Assert
● Given● When● Then
Tests as safety net for refactoring
Tests make you think about your implementation
▪ Tests often trigger refactors and even redesigns
▪ Tests make us really think about the requirements
▪ Tests make us find gaps in the requirements
Conclusion
Write unit tests to…
… reduce costs / save time… be able to confidently refactor… look at your technical design from a different angle… look at your requirements from a different angle
The Red Line
Goal of Testing
What to Test
Fixtures
Mocks
Assertions
WHAT TO TEST
Test isolated units
Keep level of collaboration the smallest possible in units under test
Test isolated units
Smallest possible
Test isolated units
Test in complete isolation
Test isolated units
No knowledge of implementation details of called method of collaborator
Test isolated units
How ?Extensive use of mocked-out collaborators in non-leaf objects/classes
Test isolated units
It results mostly in one test-class per classe.g: MyClass and MyClassTest
Test isolated units
Advantage:Less tests to writeSmaller tests to writeLess complicated setup of fixturesProbably all branches covered quickly
Boundary Cases
Test cases that cover the full spectrum of possible values.
Example 1:A method that accepts only int values between 0 and 100
public void compute(int value){}
Test with -1, 0, 10, 50, 100, 101, 'A'
Boundary Cases
Example 2:A method that removes a character in a String text
public String remove(String text, char ch)
1. Test for null text2. Test for empty text3. Test for character which is not in String4. Test for characters which comes during start, end or middle of String5. Test to cover if text String contains just one character which is equal or not equal to the
to be removed one6. Test with String contains just one character multiple times
Branches
Every time that a different path of execution can be triggered results in a different branch
Basically: if, else, for, while, do blocks and even collaborators
if(condition){
}2 branches : if true and if false
Branches: Cyclomatic complexity
Mathematical result of a formula to calculate the complexity of a piece of code.
Branches: Cyclomatic complexity
Start with a count of one for the method. Add one for each of the following flow-related elements that are found in the method.
Methods Each return that isn't the last statement of a method
Selection if, else, case, default
Loops for, while, do-while, break, and continue
Operators &&, ||, ?, and :
Exceptions catch, finally, throw, or throws clause
Threads start() call on a thread. Of course, this is a ridiculous underestimate!
Branches: Cyclomatic complexity
If this number is higher than 10 it becomes nearly impossible to test.
“Impossible to test” = impossible to have full branch coverage
Branches: Cyclomatic complexity
Keep this number low !!How ?
CLEAN CODE!!!!
Branches: Cyclomatic complexity
Where?SonarQube calculates this out of the box
Code Coverage
Expresses the amount of production code that is covered by automated tests
Code Coverage
Code Coverage: Tools
JaCoCo: replacement for Emma, fully supports Java 7 and 8, used by Sonar, EclEmma(used to be based on EMMA), Jenkins, Netbeans, IntelliJ IDEA, Gradle
Clover: Atlassian → commercial
Cobertura
Conclusion
What to test :▪ Completely in isolation▪ All Boundary cases▪ All branches
Be aware of :▪ Cyclomatic complexity
▪ SonarQube calculates it out of the box for you▪ Branch coverage
▪ Calculation possible by ▪ Jacoco, Emma, Clover, Cobertura
The Red Line
Goal of Testing
What to Test
Fixtures
Mocks
Assertions
FIXTURES
Definition (non-software)
A Fixture in non-software development is e.g.: the setup of a controlled environment to test the functioning of a piece of hardware
Definition (software)
A fixed state of software under test used as baseline for running tests = test context
Preparation of input data and setup/creation of fake or mock objects
Definition (software)
Main benefit:It avoids duplication of code necessary to initialize and clean up common
objects
Use in JUnit
From Junit 4 on there are Java 5 annotations for setting up Test fixture:@Before, @BeforeClass, @After, @AfterClass
@Before, @BeforeClass, @After, @AfterClass
@BeforeClass
public static void prepareSomeResources()
@Before
public void setUpFixtureForAllTests()
@After
public void freeResourcesAfterEachTest()
@AfterClass
public static void freeResourcesAfterAllTestsRan()
ObjectMother
ObjectMother
Factory for creating fixtures that are used in several test classes.
Catchy name thought of at thoughtworks
http://martinfowler.com/bliki/ObjectMother.html
ObjectMother
private PsychoKiller hanibal;
@Before
public void setupFixtureForHanibalLastKill2DaysAgo(){
hanibal = new PsychoKiller();
hanibal.setFirstName(“Hanibal”);
hanibal.setLastName(“Lecter”);
hanibal.setLastKillDate(current - 2 days);
}
ObjectMother
private PsychoKiller hanibal;
@Before
public void setupFixtureForHanibalLastKill2DaysAgo(){
hanibal = PsychoKillerMother.getHanibalLastKill2DaysAgo();
}
@Test
public void returns_true_if_less_than_4_days(){
assertTrue(hanibal.killedRecently());
}
Conclusion
▪ Use Fixtures to reduce code duplication
▪ In JUnit 4 : @Before @BeforeClass @After @AfterClass
▪ ObjectMother: to help keeping setup of fixtures small and concise and
in one place so reusable for other test-classes
The Red Line
Goal of Testing
What to Test
Fixtures
Mocks
Assertions
MOCKS
Why?
Mocking, Spies, Stubs, Doubles & fancy buzzwords
Isolate the code under test
Test one object, not the collaborators !
Speed up test execution
Canned replies are fast !
Make execution deterministic
Less variables means more control !
Simulate special condition
Don’t stick to the happy path.
Gain access to hidden information
Can you see The Hidden Tiger?
Our example codebase
Fancy cars !
Codebase: Car
public class Car {
public void setEngine(final Engine engine) {...}
public void start() {...}
public void stop() { … }
}
Codebase: Engine
public interface Engine {
void start();
void stop();
boolean isRunning();
}
public class DieselEngine implements Engine;
GITHUB!
https://github.com/pijalu/mocktype
Play along !
Dummy|Fake|Stubs|Spies|Mocks
Dummy
Dummy objects are passed around but never actually used. Usually they are just used to fill parameter lists
POJO Dummy
public class TestMockDummy {
final Engine dummyEngine = new DieselEngine();
Car testedCar = new Car(new DieselEngine());
@Test(expected = IllegalStateException.class)
public void testExceptionIfCarStarted() {
testedCar.start();
testedCar.setEngine(dummyEngine);
}
}
Fake
Fake objects have working implementations, but usually take some shortcut which makes them not suitable for production
POJO Fake
private final Engine fakeEngine = new Engine() { boolean started=false;
@Override
public void stop() { started=false; }
@Override
public void start() { started=true; }
@Override
public boolean isRunning() { return started; }
};
POJO Fake (cont)
…
private final Car carWithStartableEngine = new Car(fakeEngine);
@Test
public void testCarCanStart() {
carWithStartableEngine.start();
Assert.assertTrue(carWithStartableEngine.isStarted);
}
…
Stubs
Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what's programmed for the test.
POJO Stubs
private final Engine stubEngine = new Engine() { @Override
public void stop() {}
@Override
public void start() {}
@Override
public boolean isRunning() { return false; }
};
POJO Subs (cont)
private final Car car = new Car(stubEngine);
@Test
public void testCarDoesNotStartWithoutEngine() {
car.start();
Assert.assertFalse(car.isStarted);
}
Spies
NSA your calls.
POJO Spies
private class SpyEngine extends DieselEngine {
int startedCalledCount = 0;
@Override
public void start() {
startedCalledCount++;
super.start();
}
};
POJO Spies (cont)
private final SpyEngine spyEngine = new SpyEngine();
private final Car spiedTestCar = new Car(spyEngine);
@Test
public void testCarStartStartsTheEngine() {
spiedTestCar.start();
Assert.assertEquals("Engine start is called once",
1, spyEngine.startedCalledCount);
}
Mocks
Objects pre-programmed with expectations which form a specification of the calls they are expected to receive.
EasyMock: Mock (too lazy)
@RunWith(EasyMockRunner.class)
public class TestMockEasyMock
extends EasyMockSupport{
@Mock
Engine mockEngine;
…
EasyMock: Mock (cont)
@Test
public void testCarWithAMockedEngine() {
// Set the behavior
expect(mockEngine.isRunning()).andReturn(false);
mockEngine.start();
expect(mockEngine.isRunning()).andReturn(true);
mockEngine.stop();
replayAll();
EasyMock: Mock (cont 2)
// Set a car with our mocked engine
Car testedCar = new Car(mockEngine);
// Run !
testedCar.start();
testedCar.stop();
// Verify !
verifyAll();
}
POJO feels a bit ghetto ?
(Some) Existing frameworks
▪ EasyMock
▪ Mockito
▪ Unitils
▪ JMockit
And Much Much more !
Proxy Based
Class remap
Strict by default
Non-Strict by default
EasyMock
1. Arrange: Create the mock and setup behaviormock = createMock(Collaborator.class);mock.documentChanged("Document");expectLastCall().times(3);expect(mock.vote("Document")).andReturn((byte)-42);replay(mock);
3. ACT !classUnderTest.addDocument("Document", "content");
4. Assertverify(mock);
Mockito - Behavior Check
1. ArrangeList mockedList = mock(List.class);
2. ActmockedList.add("one");mockedList.clear();
3. Assertverify(mockedList).add("one");verify(mockedList).clear();
Mockito - Stubbing
1. ArrangeLinkedList mockedList = mock(LinkedList.class); when(mockedList.get(0)).thenReturn("first");when(mockedList.get(1))
.thenThrow(new RuntimeException());2. Act (examples)System.out.println(mockedList.get(0));System.out.println(mockedList.get(1));System.out.println(mockedList.get(999));
Unitils (mocking)
1. Arrange (Create the mock and setup behavior)myServiceMock =
new MockObject<MyService>(MyService.class, this);myServiceMock.returns("a value").someMethod();
2. ActmyServiceMock.getMock().someMethod();
3. Assert !myServiceMock.assertNotInvoked().someMethod();
JMockit - Expectations
@Testpublic void aTestMethod(@Mocked final MyCollaborator mock){ new NonStrictExpectations() {{ mock.getData(); result = "my test data"; mock.doSomething(anyInt, "some expected value", anyString);
times=1; }}; // In the replay phase, the tested method would call the "getData" and "doSomething" // methods on a "MyCollaborator" instance. ...
// In the verify phase, we may optionally verify expected invocations to // "MyCollaborator" objects. ...
}
JMockit - Verification
... new Verifications() {{ // If no new instance of the mocked class should have been // created with the no-args constructor, we can verify it: new MyCollaborator(); times = 0; // Here we verify that doSomething() was executed at least once: mock.doSomething(); // Another verification, which must have occurred no more than three // times: mock.someOtherMethod(
anyBoolean, any, withInstanceOf(Xyz.class)); maxTimes = 3; }};}
Conclusion
▪ Mocking helps you to create better tests▪ Mocking Framework(s) help keeping overhead low
The Red Line
Goal of Testing
What to Test
Fixtures
Mocks
Assertions
ASSERTIONS
Assertions
What ? Verify if the result of the test is what we expected.
Assertions
How ? Should be automated, no human intervention necessary No logging, System.out.println() etc. to be used Use framework(s) to achieve this.
Junit assertions
TypesassertEquals, assertSame, assertTrue, assertFalse, assertNull, assertNotNull
Advantages Most often used, so best known.
Junit assertions
Drawbacks▪ Not always very readable
assertEquals(expected, result) or assertEquals(result, expected)
▪ Number of assertions are limited▪ Comparing (complex) objects is hard.
impossible when object has not equals() implemented !
ConclusionIt’s ok to use them for primitive types.But there are better alternatives.
Hamcrest assertions
What ?▪ Hamcrest is a framework for writing matcher objects allowing ‘match’
rules to be defined declaratively.assertThat(Object, Matcher<T>);
▪ Can easily be integrated with other frameworks like Junit, TestNG, Mockito, EasyMock, Jmock,...
Hamcrest assertions
Advantages▪ Improved readability of tests
assertThat(ObjectToBeChecked, equalTo(OtherObject))
assertThat(ObjectToBeChecked, is(equalTo(OtherObject));
assertThat(collection, hasSize(2);
▪ Better failure messageassertThat(3, greaterThan(5));
Expected: a value greater than <5>
but: <3> was less than <5>
Hamcrest assertions
▪ Combination of MatchersAllows to assert more precisely.
assertThat(array, not(emptyArray());
assertThat(collections, everyItem(greaterThan(10));
▪ Write your own Matcher
This can occur when you find a fragment of code that test the same set of properties over and over again and you want to bundle the fragment into a single assertion.
But be aware, there are already plenty of matchers available, make sure you are not writing existing code again.
Hamcrest assertions
Example : To test if a double has a value NaN (not a number)
Test we want to write :
public void testSquareRootOfMinusOneIsNotANumber () {
assertThat(Math.sqrt(-1), is(notANumber()));
}
public class IsNotANumber extends TypeSafeMatcher<Double> {
@Override
public boolean matchesSafely(Double number) {
return number.isNaN();
}
public void describeTo(Description description) {
description.appendText("not a number");
}
@Factory
public static <T> Matcher<Double> notANumber() {
return new IsNotANumber();
}
}
Hamcrest assertions
Hamcrest assertions
▪ More possibilities to compare objects
It is possible to check objects that don’t have equals() implemented.
assertThat(ObjectToBeChecked, samePropertyValuesAs(OtherObject));
but not possible for objects with composition !
Better use ReflectionAssert.assertReflectionEquals of unitils !
Hamcrest assertions
Drawbacks▪ Finding the right Matcher
The matchers are not set in one place. Most matchers are accessible via the Matcher class, but some are located in the CoreMatcher class, and some are in another package.
example:
hasItem() : Matcher classhasItems() : IsCollectionContaining class
Other frameworks
Unitils▪ ReflectionAssert
This assertion loops over all fields in both objects and compares their values using reflection. If a field value itself is also an object, it will recursively be compared field by field using reflection. The same is true for collections, maps and arrays.
▪ Lenient assertionsAdding some levels of leniency to the ReflectionAssert checks. (order list, ignoring defaults, dates, assertLenientEquals)
▪ Property assertionsMethods to compare a specific property of two objects
assertPropertyLenientEquals("id", 1, user);
assertPropertyLenientEquals("address.street", "First street", user);
Other frameworks
Fest (http://code.google.com/p/fest/)
Supports both Junit and TestNG
assertThat(collection).hasSize(6).contains(frodo, sam);
Assertions
Misuse of assertions▪ Manual assertions
This practice misses out the main benefits of testing automation — the ability to continuously run the tests in the background without intervention
■ Multiple assertions
■ Redundant assertionsExtra calls to an assert method where the condition being tested is a hard coded valueassertTrue(“always true”, true)
■ Using the wrong assertionsassertTrue("Object must be null", actual == null);assertTrue("Object must not be null", actual != null);
Assertions
What about void methods ?Often if a method doesn't return a value, it will have some side effect. There may be a way to verify that the side effect actually occurred as expected.
Especially exception testing should not be forgotten.
Assertions
MyClass {
public void addElement(String element, List<String> elements) {
elements.add(element);
}
}
public void testAddElement() {
List<String> elements = new ArrayList();assertEquals(0, elements.size() );myClassTest.addElement(“test”, elements);assertEquals(1, elements.size() );
}
Conclusion
▪ Always make sure your assertions are fully automated
▪ Junit assertions are ok for primitive types
▪ Hamcrest offers a lot of interesting matchers that allows you to assert
more precise
▪ Unitils is a better alternative when comparing objects
▪ You can use them all together !
The Red Line
Goal of Testing
What to Test
Fixtures
Mocks
Assertions
Question ?
PRACTICAL
Exercises !
github: https://github.com/pijalu/utdemo
▪ Master: Default start
▪ Branches: Different solutions with different mocking framework…
Don’t look ;-)
Factoid - Model
Fact
▪ Simple POJO▪ Stores a fact▪ It’s a string
Factoid - Provider
Provider interface:▪ Provides a list of fact to a client
▪ int size(): Number of facts in the provider▪ Fact getFact(index): Return a fact (0->size-1)
▪ Implementation:▪ FileFactProvider
▪ Loads facts from a file.▪ Line oriented
Factoid- Service
FactService interface:▪ Returns a fact to client:
▪ Fact getAFact(): Return a fact
▪ Implementation:▪ RandomFactService:
▪ Returns a random fact using a provider▪ Uses Random▪ Builds an array to avoid repetition/ensure all facts are returned
Factoid - Main
Factoid main class
▪ Loads a File with a (File)FactProvider
▪ Loads a (Random)FactService using created fact provider
▪ Calls FactService getAFact()
Factoid - What to do
▪ Select the mocking framework you want▪ EasyMock to start !
▪ Check the FIXME in the existing code
▪ Fix as many as you can !▪ Be creative
▪ Look for other issues ;-)
GO !
Thanks for your attendance