Agile Development & Companies

• Harvard Business Review: “Why the lean start-up changes everything”

• Agile development applied to entire business of start-ups

Iterate!

Simplest possible

prototype!

Image: leadnet.org

Testing

Vaughn Betz ECE 297

Testing Philosophy

• “If you didn’t test it, it doesn’t work”– Assume all code broken until proven otherwise– Look at what the program does not what it is

supposed to do• Scientist: you are testing hypotheses about program

– Think of every case your program should handle cover them all

Testing Time

• Develop tests as or before you code– Test-driven development

Test & Evaluate

Refine

Prototype

Test-Driven Development

• The tests are the detailed specification– And now can be executed, not just read

Less specification document

More tests

Types of Tests

Unit Tests

End to End Tests

End-to-end (system) tests• Test entire system (like end user would)• Hard to debug a failure problem could be anywhere!• Fragile: often test some exact user input would

produce some exact output text / file

Unit Tests• Test small pieces of system (single classes / functions)• Much easier to debug a failure start with that class /

function• Don’t check user interface (I/O): check API / class

does what it should with code (testfixture)

Types of Tests

Unit Tests

Integration Tests

End to End Tests

Integration Tests• Bigger unit tests• Work the same way• But now test multiple classes / bigger functionality• Moderate difficulty to debug

Testing Automation

• You will re-test often– Every time you change the program– How to speed up?

• System tests: save in files– Use file redirection

• prog.exe < in.txt > out.txt • diff out.txt out_good.txt // matched known good result?

– Less fragile• Write validity checker instead of checking exact output match

• Write scripts to run all tests and check results

Unit Tests

• Want:– To test individual classes / functions– How?

• Carefully chosen user input?

struct Point { float x; float y; t_point& operator*=(float rhs); . . .

Output ...

No!

Unit Tests

• Write new code – To put class in right state– To directly send it some input / make some calls– To immediately check the responses– Test driver

Unit Tests

struct Point { float x; float y; t_point& operator*=(float rhs); . . .

int test1 () { Point testme (1, 2); testme *= 3; if (testme != Point (3, 6)) { cout << “uh – oh” << endl; return (1); } return (0);}

myCode.cpp

tester.cpp

int main () { int error = 0; error += test1(); . . . // Lots more tests

test_main.cpp

g++ myCode.cpp tester.cpp test_main.cpp –o test.exe

g++ myCode.cpp main.cpp –o prog.exe

Unit Test Frameworks• Lots of repetitive code to create test drivers

– Set up the test– Check if the test passed– Run all the tests– Output appropriate messages– Collect statistics

• Unit Test Frameworks– Useful macros (#defines) and functions to simplify

coding– We are using UnitTest++

• Powerful, but easy to learn• ECE 297 Unit Test Quick Start Guide

TEST(..) or TESTFIXTURE (…)

CHECK(..)

RunAllTests()

Automatic

Automatic

“Test the Seams”

• Overlapping tests good– Your code + partner’s code

• Both unit tested

– Make sure there’s an integration test

• Therac-25

Testing Tools

• What tools?

1.Debugger– Use to debug when a test fails– Use to verify new code

• Step through it and watch execution

2.Memory checker– Program seg faults?– Program behaving very strangely?– Maybe you are accessing memory you shouldn’t be!– Run valgrind

Testing Tools

3. Code coverage– Tools that can track what lines of your programs have

executed over all your tests

int someFunc (int input) { if (input == 0) return (3); else return (7);}

MyCode.cpp

int main () { int j = someFunc (8); // Wow I’m bad at testing!}

main.cpp

Testing Tools

3. Code coverage– Tools that can track what lines of your programs have

executed over all your tests

int someFunc (int input) { if (input == 0) return (3); else return (7);}

MyCode.cpp

int main () { int j = someFunc (8); // Wow I’m bad at testing!}

main.cpp

No test reaches this line.Code coverage: 6 out of 7 lines or 86%

Google: Testing so Important, Tutorials in the Bathroom

“Testing on the Toilet”

Good Coding Style

1. Use White Space

White space: show code organization– Indent properly (3 or 4 spaces) per { }.– Leave blank lines between functions / key blocks

int sumVec (int vec[], int nElem) {int i, result = 0;for (i = 0; i < nElem; i++) {result += vec[i];}return (result);}void nextFunc (int i) {…

int sumVec (int vec[], int nElem) { int i, result = 0; for (i = 0; i < nElem; i++) { result += vec[i]; } return (result);}

void nextFunc (int i) {

float di (float a, float b)

{

float val, d, x, x2, y;

d = 1.e-4;

val = 0;

for (x = a; x < b; x += d) {

x2 = x + d;

if (x2 > b)

x2 = b;

y = 0.5 * ((1. / x) + (1. / x2));

val += y * (x2 - x);

}

return (val);

}

What does this do?

float definite_integral (float x_left, float x_right)

{

float integral, step_size, x1, x2, y_average;

step_size = 1.e-4;

integral = 0;

for (x1 = x_left; x1 < x_right; x1 += step_size) {

x2 = x1 + step_size;

if (x2 > x_right)

x2 = x_right;

y_average = 0.5 * ((1. / x1) + (1. / x2));

integral += y_average * (x2 – x1);

}

return (integral);

}

What does this do?

2. Descriptive Variable Names

• Use descriptive names– Variables, functions, structs/types, …

• get_file_name ( ); // Use _ to separate• getFileName (); // Or use upper case to mark words

• Types: start with a capital letter• Variables: start with lowercase

– class MyClass { …– MyClass oneVar;

float definite_integral (float x_left, float x_right)

{

float integral, step_size, x1, x2, y_average;

step_size = 1.e-4;

integral = 0;

for (x1 = x_left; x1 < x_right; x1 += step_size) {

x2 = x1 + step_size;

if (x2 > x_right)

x2 = x_right;

y_average = 0.5 * ((1. / x1) + (1. / x2));

integral += y_average * (x2 – x1);

}

return (integral);

}

What does this do?

// Compute the definite integral of 1/x between x_left and x_right via the

// trapezoidal method. Smaller values of step_size improve accuracy, but

// increase computation time.

float definite_integral_of_one_over_x (float x_left, float x_right)

{

float integral, step_size, x1, x2, y_average;

step_size = 1e-4;

integral = 0.;

for (x1 = x_left; x1 < x_right; x1 += step_size) {

x2 = x1 + step_size;

if (x2 > x_right) // in case (x_right – x_left) is not a multiple of step_size

x2 = x_right;

y_average = 0.5 * ((1. / x1) + (1. / x2)); // average of y(x1) and y(x2)

integral += y_average * (x2 – x1);

}

return (integral);

}

Comment what whole function does

Comment any tricky bits of code

Comments: Usefulness?/* Functions to simulate transistors. * We model transistors as nonlinear elements, * by looking up the source-drain current for each V … */

trans_sim.cpp

/* Main data structure used to store all information * about a U of T student. Linked list. */struct StudentRecord { int nClassesCurrent; // Number of classes enrolled in. int nClassesComplete; // Number of completed classes, // not including currently enrolled ones StudentRecord *next; // Pointer to next (linked list) record

StudentRecord.h

// Compute the sum of the array, over all its elementsint sum = 0;for (int i = 0; i < nElem; i++) sum += array[i];

program.cpp

3. “High-Level” Comments• Most important comments: give the big picture

– Documentation should be in the comments– Not a separate document will get out of date

1. Top of files // Functions to simulate transistors. We proceed in 6 stages …

2. Class / data structure definitions– Understand the data can understand the program!

3. Start of functions

4. Tricky code

Not very useful comments:– Translate C++ to English

Most important

Leastimportant

Thoughts on This Code?int checkWeights (int weights[20]) { for (int i = 0; i < 20; i++) { if (weights[i] < 0) return (-1); if (weights[i] == 0) return (0); } return (1);}

1. Using a “magic number”: 20– Change array size: must find and change all 20’s

2. Returning magic numbers: -1, 0, 1– Must read code carefully to see what each means

4. Use Named Constantsconst int NUM_WEIGHTS = 20; // 1. Constant variable#define WEIGHT_ZERO 0 // 2. Pre-processor constantenum WtReturn {HAS_NEG = -1, HAS_ZERO = 0, ALL_POS = 1}; // 3. make an “enumeration” (list) of int constants

int checkWeights (int weights[NUM_WEIGHTS]) { for (int i = 0; i < NUM_WEIGHTS; i++) { if (weights[i] < 0) return (HAS_NEG); if (weights[i] == 0) return (HAS_ZERO); } return (ALL_POS);}• Three ways to make constants use any way you like• Name: ALL CAPITALS (convention)