Help, my tests are killing me!!

HELPMY Tests ARE KILLING ME!!

brian swan @bgswanfollow meon twitter

Contains code some people may find offensive

Testsuite: org.springframework.util.StopWatchTestsTests run: 1, Failures: 1, Errors: 0, Time elapsed: 0.177 sec

Testcase: testValidUsage took 0.171 sec FAILEDUnexpected timing 167junit.framework.AssertionFailedError: Unexpected timing 167 at org.springframework.util.StopWatchTests.testValidUsage(StopWatchTests.java:48)

public void testValidUsage() throws Exception { StopWatch sw = new StopWatch(); long int1 = 166L; long int2 = 45L; String name1 = "Task 1"; String name2 = "Task 2";

assertFalse(sw.isRunning()); sw.start(name1); Thread.sleep(int1); assertTrue(sw.isRunning()); sw.stop();

long fudgeFactor = 0L; assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() >= int1); assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() <= int1 + fudgeFactor); sw.start(name2); Thread.sleep(int2); sw.stop(); assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() >= int1 + int2); assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() <= int1 + int2 + fudgeFactor);

assertTrue(sw.getTaskCount() == 2); String pp = sw.prettyPrint(); assertTrue(pp.indexOf(name1) != -1); assertTrue(pp.indexOf(name2) != -1);

StopWatch.TaskInfo[] tasks = sw.getTaskInfo(); assertTrue(tasks.length == 2); assertTrue(tasks[0].getTaskName().equals(name1)); assertTrue(tasks[1].getTaskName().equals(name2)); sw.toString(); }

AARGGHH



// TODO are timings off in JUnit? Why do these assertions sometimes fail // under both Ant and Eclipse?

//long fudgeFactor = 5L; //assertTrue("Unexpected timing " + sw.getTotalTime(), sw.getTotalTime() >= int1); //assertTrue("Unexpected timing " + sw.getTotalTime(), sw.getTotalTime() <= int1 + fudgeFactor); sw.start(name2); Thread.sleep(int2); sw.stop(); //assertTrue("Unexpected timing " + sw.getTotalTime(), sw.getTotalTime() >= int1 + int2); //assertTrue("Unexpected timing " + sw.getTotalTime(), sw.getTotalTime() <= int1 + int2 + fudgeFactor);



https://github.com/SpringSource/spring-framework/blob/master/spring-core/src/test/java/org/springframework/util/StopWatchTests.java





http://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_Readability_Test


The UnREADABLE TEST

produced by Long Test Methods featuring Undecipherable Failures

introducing Developer Confusion and Wasted Developer Time

Flesch Reading Ease

http://en.wikipedia.org/wiki/Flesch–Kincaid_Readability_Test

http://en.wikipedia.org/wiki/Flesch

http://en.wikipedia.org/wiki/Flesch



MORE READABILITY

Produced By SMALL WORDS and SHORT SENTENCES

http://www.arrestedcomputing.com/pubs/readability-issta.pdf










THELINE LENGTH IDENTIFIER MASSACRE

“

”

THE INCREDIBLE

it "adds items to top" do stack = Stack.new # Setup stack.add "thing" # Exercise assert_equal "thing", stack.top # Verify stack = nil # Teardown end

it "adds items to top" do stack = Stack.new # Setup stack.add "thing" # Exercise assert_equal "thing", stack.top # Verify end

it "adds items to top" do stack = Stack.new # Arrange stack.add "thing" # Act assert_equal "thing", stack.top # Assert end

it "adds items to top" do stack = Stack.new # Given stack.add "thing" # When assert_equal "thing", stack.top # Then end

before do @stack = Stack.new # Given end it "adds items to top" do @stack.add "thing" # When assert_equal "thing", @stack.top # Then end

before do @stack = Stack.new # Given end it "is initially empty" do assert_empty @stack # Then end

it "returns the total timesheet hours worked of the timesheet" do timesheet = Timesheet.new task_items = [stub(start_time: 1359291600, end_time: 1359295200), stub(start_time: 1359295200, end_time: 1359298900)] timesheet.stub(:task_items) { task_items } timesheet.calculate_total_hours.should == 2.0 end

http://www.superpumpup.com/sub-ms-test






Extract Given

You have a long, confusing or poorly named test setup.

Turn the fragment into a method or member variable whose name describes the “given”.

http://martinfowler.com/bliki/ObjectMother.html





http://nat.truemesh.com/archives/000714.html





def timesheet_with_two_one_hour_tasks timesheet = Timesheet.new task_items = [stub(start_time: 1359291600, end_time: 1359295200), stub(start_time: 1359295200, end_time: 1359298900)] timesheet.stub(:task_items) { task_items } timesheet end

it "returns the total timesheet hours worked of the timesheet" do timesheet_with_two_one_hour_tasks.calculate_total_hours.should == 2.0 end

def timesheet_with_two_one_hour_tasks timesheet = Timesheet.new task_items = [stub(start_time: 1359291600, end_time: 1359295200), stub(start_time: 1359295200, end_time: 1359298900)] timesheet.stub(:task_items) { task_items } timesheet end


Extract Collaborator

An object your test depends on is confusing or poorly named.

Extract the collaborating object into a method or member variable whose name describes its purpose.

def one_hour_task stub(start_time: 1359291600, end_time: 1359295200) end

def timesheet_with_two_one_hour_tasks timesheet = Timesheet.new task_items = [one_hour_task, one_hour_task] timesheet.stub(:task_items) { task_items } timesheet end


def one_hour_task stub(start_time: 1359291600, end_time: 1359295200) end

def timesheet_with_two_one_hour_tasks timesheet = Timesheet.new task_items = [one_hour_task, one_hour_task] timesheet.stub(:task_items) { task_items } timesheet end


Inline TempYou have a temp that is assigned to once with a simple expression, and the temp is getting in the way of other refactorings.

Replace all references to that temp with the expression.

http://www.refactoring.com/catalog/inlineTemp.html





def one_hour_task stub(start_time: 1359291600, end_time: 1359295200) end def timesheet_with_two_one_hour_tasks timesheet = Timesheet.new timesheet.stub(:task_items) { [one_hour_task, one_hour_task] } timesheet end


def one_hour_task stub(start_time: 1359291600, end_time: 1359295200) end def timesheet_with_two_one_hour_tasks timesheet = Timesheet.new timesheet.stub(:task_items) { [one_hour_task, one_hour_task] } timesheet end


Use meaningful data

Your test uses arbitrary literal values as test data.

Replace arbitrary values with values that have meaning in the domain of the test.

ONE_HOUR_IN_SECONDS = (60*60*1)

def one_hour_task stub(start_time: 0, end_time: ONE_HOUR_IN_SECONDS) end

def timesheet_with_two_one_hour_tasks timesheet = Timesheet.new timesheet.stub(:task_items) { [one_hour_task, one_hour_task] } timesheet end


ONE_HOUR_IN_SECONDS = (60*60*1)

def one_hour_task stub(start_time: 0, end_time: ONE_HOUR_IN_SECONDS) end



Extract Test

Your test method is checking multiple behaviours, or checking behaviour of a collaborating object.

Extract a new test method for each behaviour.

describe Task do

it "knows duration in hours" do task = Task.new(start_time: 0, end_time: 3600) task.duration.should == 1.0 end end

def one_hour_task stub(duration: 1) end



it "returns the total timesheet hours worked of the timesheet" do one_hour_task = stub(duration: 1) timesheet = Timesheet.new timesheet.stub(:task_items) { [one_hour_task, one_hour_task] }

timesheet.calculate_total_hours.should == 2.0 end







it "returns the total timesheet hours worked of the timesheet" do one_hour_task = stub(duration: 1) timesheet = Timesheet.new timesheet.stub(:task_items) { [one_hour_task, one_hour_task] }

timesheet.calculate_total_hours.should == 2.0 end

IIThe UnREADABLE TEST






X...the Unknowntest case

“clarity trumps brevity in test names

https://www.facebook.com/notes/kent-beck/shorts-not-always-sweet-the-case-for-long-test-names/564493423583526







Rename Method

The name of a method does not reveal its purpose.

Change the name of the method.

http://www.refactoring.com/catalog/renameMethod.html





public void testIsRunningWhenStartedAndTimeMultipleTasksAndTaskCountAndPrettyPrintAndTaskInfo() throws Exception { StopWatch sw = new StopWatch(); long int1 = 166L; long int2 = 45L; String name1 = "Task 1"; String name2 = "Task 2";





public void testTimeMultipleTasks() throws Exception { StopWatch sw = new StopWatch(); long int1 = 166L; long int2 = 45L; String name1 = "Task 1"; String name2 = "Task 2";

sw.start(name1); Thread.sleep(int1); sw.stop();

long fudgeFactor = 5L; assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() >= int1); assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() <= int1 + fudgeFactor); sw.start(name2); Thread.sleep(int2); sw.stop(); assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() >= int1 + int2); assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() <= int1 + int2 + fudgeFactor); }

public void testTimeMultipleTasks() throws Exception { StopWatch sw = new StopWatch(); StopWatch.TaskInfo task1 = new StopWatch.TaskInfo("Task 1", 166L); StopWatch.TaskInfo task2 = new StopWatch.TaskInfo("Task 2", 45L);

sw.start(task1.getTaskName()); Thread.sleep(task1.getTimeMillis()); sw.stop();

long fudgeFactor = 10L; assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() >= task1.getTimeMillis()); assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() <=

task1.getTimeMillis() + fudgeFactor); sw.start(task2.getTaskName()); Thread.sleep(task2.getTimeMillis()); sw.stop(); assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() >= task1.getTimeMillis() + task2.getTimeMillis()); assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() <= task1.getTimeMillis() + task2.getTimeMillis() + fudgeFactor); }


timeTask(sw, task1);

long fudgeFactor = 10L; assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() >= task1.getTimeMillis()); assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() <= task1.getTimeMillis() + fudgeFactor);

timeTask(sw, task2);

assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() >= task1.getTimeMillis() + task2.getTimeMillis()); assertTrue("Unexpected timing " + sw.getTotalTimeMillis(), sw.getTotalTimeMillis() <= task1.getTimeMillis() + task2.getTimeMillis() + fudgeFactor); }

private void timeTask(StopWatch sw, StopWatch.TaskInfo task) throws Exception { sw.start(task.getTaskName()); Thread.sleep(task.getTimeMillis()); sw.stop(); }

Replace Assertion

Your test method uses a basic assertion to check for a specific condition.

Use a more specific assertion provided by your test framework, or create a custom assert method.


timeTask(sw, task1); assertElapsedTime(sw, task1.getTimeMillis());

timeTask(sw, task2); assertElapsedTime(sw, task1.getTimeMillis() + task2.getTimeMillis()); }

private void assertElapsedTime(StopWatch sw, int expectedDuration) { double tolerance = 10.0; assertThat((double)sw.getTotalTimeMillis(), is(closeTo((double)expectedDuration, tolerance))); }

private void timeTask(StopWatch sw, StopWatch.TaskInfo task) throws Exception { sw.start(task.getTaskName()); Thread.sleep(task.getTimeMillis()); sw.stop(); }







timeTask(sw, task1); assertElapsedTime(sw, task1.getTimeMillis());

timeTask(sw, task2); assertElapsedTime(sw, task1.getTimeMillis() + task2.getTimeMillis()); }

Readability Short Test Methods

Shorter Line Length

Fewer Identifiers

Clear Test Method Names

Refactorings Extract Given

Extract Collaborator

Use Meaningful Data

Extract Test

Replace Assertion

Technology

Help, my tests are killing me!!