Embed Size (px)
DESCRIPTION
Does investing in high quality developer test code really matter? And does it help a team to become more productive?
Citation preview
First Steps in Testing Analytics: Does Test Code Quality Matter?
Andy ZaidmanDelft University of Technology
Nederlandse Testdag17 november 2014, Amersfoort
Joint work with Moritz Beller & Georgios Gousios(TU Delft)
Dimitrios Athanasiou, Ariadi Nugroho & Joost Visser(Software Improvement Group)
Who am I?
Software engineering teacher by day
Testing evangelist
Software engineering researcher by night
What would you like to know?What analytics?
• ???
WHATEVER HAPPENS…
@azaidman
TODAY
2 test analytics stories
STORY 1
Test Code Quality Matters
Andy teaching
First year programming course @ TU Delft
In week 3 everyone is unit testing
“I don’t have time for testing”
“I don’t like testing, my code is good anyway”
“How much should I test?”
“How good should my test code be?”
@Test
public void testConstructor() {
assertTrue(true)
}
Solution 1
Solution 2
@Test
public void testConstructor() {
House h = new House(“Street”, “9”);
assertEquals(“Street 9”, h.toString());
}
Solution 3
@Test
public void testConstructor() {
House h = new House(“Street”, “9”);
assertEquals(“Street”, h.getStreet());
assertEquals(9, h.getNumber());
}
Solution 4
@Testpublic void testConstructorStreet() {
House h = new House(“Street”, “9”);assertEquals(“Street”, h.getStreet());
}
@Testpublic void testConstructorNumber() {
House h = new House(“Street”, “9”);assertEquals(9, h.getNumber());
}
My first reaction?
SHOOT FIRST
BRINGS US
NOWHERE
Analytics?
Testing Analytics!
Thou shalt not have less than 70% coverage
What makes a good
unit test?
What makes a good
integration test?
What makes a good
developer test?
What makes a good
developer test?
Complete
Effective
Maintainable
Complete
• How much of the code is covered by the tests?
• How many of the decision points in the code are tested?
Code coverage
#assertions/cyclomatic complexity
Effective
• How able is the test code to detect defects?
• How able is the test code to locate the cause?
Assertion density #assertions / LOCtest
Directness unit versus integration testing
Maintainable
• How maintainable is the system’s test code?
Software Improvement Group quality model
Test Code Quality Model*
Complete
+ Effective
+ Maintainable
* Calibrated using 86 systems
SO
WHAT?
Bettertests
Easier development
?
High quality tests do not lead to quicker/easier bug fixing
Why?
The low-hanging fruit is already solved if you have good tests… the difficult bugs remain
Observation 1*
* While studying 75 years of combined development of 18 software systems
High quality tests do lead to higher productivity
• More issues fixed per month per developer
• More issues fixed per month per KLOC
Why?
Implementing new features does seem to benefit from having a safety net for existing
functionality.
Observation 2
High quality tests do make your team more productive!
Action(able)
The single most important thing for test
code quality
#assertionsMcCabe cyclomatic complexity
≥ 1
…
STORY 2
Testing takes 50% of your time
Says this guy
• Study from 1975
• Questionable research approach
Differences?
• Waterfall
• Developer testing
• Programming language
• Tools (e.g. IDE)
New research
+ +
Recording
1. Time spent on developing production code
2. Time spent on developing test code
3. How frequently are Junit tests executed?
4. How frequently do these test runs fail?
124 developers participating
You might win
http://testroots.org/testroots_watchdog.html
…
My ultimate dream?
Continuous
background
testing
Take home
• Test code quality matters for the productivity of your team
• #assertions/cyclomatic complexity ≥ 1
• Help us out with our research!
http://azaidman.wordpress.com
httpp://www.st.ewi.tudelft.nl/~zaidman
@azaidman
