Test in Action – Week 4Good Tests

Hubert Chan

Test Lint

• Test Lint– Trustworthiness– Maintainability– Readability

• References– Test Lint (http://docs.typemock.com/lint/ )

Trustworthy Tests

• Trustworthy Tests– Always Asserts Something– Avoid unpredictable factor– Don’t depend on other tests– Avoid Logic in Tests

Always Asserts Something

• Always Asserts Something– Assertion means “Verification”– Unit Tests need to “Check Something”

• Exceptions– Check exception not thrown• Name “Login_Call_NoExceptionThrown”

– Specified Mock Object

Avoid Unpredictable Factor

• Avoid Unpredictable Factor– Unpredictable Factor• Random Number• Date Time

– Cause• Unreliable/Inconsistent Result• Hard to write Assertion statement

Avoid Unpredictable Factor

• Solution– Use Fake– Use hard-code values

$pseudoRandom = 1234;$this->assertEquals(1, sut.DoSomething(pseudoRandom));

$random = rand();$this->assertEquals(1, sut.DoSomething($random));

Don’t Depend on Other Tests

• Exampleclass LogAnalyzerDependTest extends PHPUnit_Framework_TestCase {

public function test_LogAnalyzerDepend_Construct_NoException() { $this->analyzer = new LogAnalyzerDepend(); $this->analyzer->initialize(); }

public function test_IsValid_InValidContent_ReturnFalse() { $this->test_LogAnalyzerDepend_Construct_NoException(); $this->assertFalse($this->analyzer->is_valid('abc')); }}

Don’t Depend on Other Tests

• Symptom– A test calls another tests– It requires other tests to create/delete objects– A test depends on system state set by other tests– Test order matters

Don’t Depend on Other Tests

• Why not?– Cannot provide explicit testing result– Implicit tests flow– The testA failed because callee testB failed

• Solution– Extract reused code to utility– For create/delete object, use “setUp” and

“tearDown” instead

Avoid Logic in Tests

• Test more than one thing– Number of Assertion > 1– Logics better not in tests• switch, if, or else statement• foreach, for, or while loops

Avoid Logic in Tests- Example

• Test Codepublic function test_ImplodeAndExplode_ValidContent_CorrectResult() { $instance = new MoreThanOne(); $test_array = array('1', '2', '3'); $test_string = '1,2,3';

for ($i = 0; $i < 2; $i++) { if ($i === 0) { // Test explode2 $result = $instance->explode2(',', $test_string); $this->assertEquals($test_array, $result); } elseif ($i === 1) { // Test implode2 $result = $instance->implode2(',', $test_array); $this->assertEquals($test_string, $result); } }}

Make Clean Tests

• Change your tests– Removing invalid tests– Renaming / Refactoring tests– Eliminating duplicate tests

Trustworthiness – Do it

• Do it– Testing only one thing– Keep safe green zone• No dependency to real database/network• Keep result consistent

– Assuring code coverage– Attitude• Add a unit test for newly tickets/trackers

Maintainable Tests

• Maintainable Tests– Test public function only– Don’t Repeat Yourself• Use Factory Function over Multiple Object Creation• Use setUp

– Using setUp in a maintainable manner– Avoid Over Specification in Tests– Avoid multiple asserts

Test Public Function Only

• Test Public Function Only– Design/Program by Contract• Private/Protected method might be changed

– Extract private/protected function to new class• Adopt Single Responsibility Principle (SRP)

Semantic Change

• Semantic Change is really a PAIN– API change may break all tests– Each test need to be changed

public function test_IsValid_InValidContent_ReturnFalse_New() { $analyzer = new LogAnalyzer(); $analyzer->initialize(); // new requirement $this->assertFalse($analyzer->is_valid('abc'));}

Use Factory Function over Multiple Object Creation

• Use a factory functionprotected function create_LogAnalyzer() { // Factory Method $analyzer = new LogAnalyzer(); $analyzer->initialize(); // New API handled here return $analyzer;}public function test_IsValid_InValidContent_ReturnFalse() { $analyzer = $this->create_LogAnalyzer(); // Use factory $this->assertFalse($analyzer->is_valid('abc'));}

Use setUp over Multiple Object Creation

• Use setUpprotected function setUp() { // setUp method $this->analyzer = new LogAnalyzer(); $this->analyzer->initialize(); // New API handled here}

public function test_IsValid_InValidContent_ReturnFalse() { $this->assertFalse($this->analyzer->is_valid('abc'));}public function test_IsValid_ValidContent_ReturnFalse() { $this->assertTrue($this->analyzer->is_valid('valid'));}

Maintainable setUp

• Maintainable setUp– setUp() should be generic• Cohesion• Initialized object should be used in all tests• Might not be appropriate to arrange mock/stub in

setUp()

– setUp() should be kept his readability

Avoid Over Specification in Test

• Over specified in test– Verify internal state/behavior of object– Using mocks when stubs are enough– A test assumes specific order or exact string

matches when it isn’t required.

Verify internal state/behavior of object

• Solution– Never verify internal state/behavior– Maybe no need to testpublic function test_Initialize_WhenCalled_SetsDefaultDelimiterIsTabDelimiter(){ $analyzer = new LogAnalyzer(); $this->assertEquals(null, $analyzer->GetInternalDefaultDelimiter() ); $analyzer->Initialize(); $this->assertEquals('\t', $analyzer->GetInternalDefaultDelimiter() );}

Avoid Multiple Asserts

• Why not?– Assertion failure will throw exception. Multiple

assertion cannot get all failure point at once– Test multiple thing in one tests

• Solution– Separate tests for different assertion– Use data provider / parameter tests

Data Provider Sample

class DataTest extends PHPUnit_Framework_TestCase { /** * @dataProvider provider */ public function testAdd($a, $b, $c) { $this->assertEquals($c, $a + $b); }

public function provider() { return array( array(0, 0, 0), array(0, 1, 1), array(1, 0, 1), array(1, 1, 3) ); }}

• Test Code

Readable Tests

• Test Naming• Variable Naming• Good Assert Message• Separate Arrange and Assertion• Mock and Stub Naming

Test Naming

• Function Name– Test function name should be

test_<function>_<scenario>_<expect_behavior>– Example• test_escape_evenBackSlashesData_successEscape

Variable Name

• Avoid Hard Code in testspublic function test BadlyNamedTest() { $log = new LogAnalyzer(); $result= log.GetLineCount("abc.txt"); $this->assertEquals(-100, result);}

public function test WellNamedTest() { $log = new LogAnalyzer(); $COULD_NOT_READ_FILE = -100; $result= log.GetLineCount("abc.txt"); $this->assertEquals($COULD_NOT_READ_FILE, result);}

Good Assertion Message

• Good Assertion Message– Don’t repeat what the built-in test framework

outputs to the console.– Don’t repeat what the test name explains.– If you don’t have anything good to say, don’t say

anything.– Write what should have happened or what failed

Separate Arrange and Assertion

• Separate Arrange and Assertion

public function test_BadAssertMessage() { $this->assertEquals(COULD_NOT_READ_FILE, log->GetLineCount("abc.txt") ); }

public function test_GoodAssertMessage() { $result = log->GetLineCount("abc.txt"); $this->assertEquals($COULD_NOT_READ_FILE, $result); }

Mock and Stub Naming

• Include “mock” and “stub” in variable name public function test_sendNotify_Mock_NoException() {

$notify_content = 'fake_content'; $mock_notifier = $this->getMock('NotifierInterface'); $mock_notifier->expects($this->once()) ->method('notify') ->with($this->anything(), $this->equalTo($notify_content));

$alert_system = new AlertSystem( $mock_notifier, $stub_provider ); $alert_system->send_notify('Alert!!'); }

Q & A

PHPUnit and Selenium

• Use PHPUnit to do– Integration Test– Acceptance Test

• References– PHPUnit Selenium• http://

www.phpunit.de/manual/current/en/selenium.html• http://seleniumhq.org/documentation/

PHPUnit and Selenium

• Use PHPUnit and Seleniumclass WebTest extends PHPUnit_Extensions_SeleniumTestCase { protected function setUp() { $this->setBrowser('*firefox'); $this->setBrowserUrl('http://www.example.com/'); }

public function testTitle() { $this->open('http://www.example.com/'); $this->assertTitle('Example WWW Page'); }}

PHPUnit and Selenium