History of Software Testing

Philippe CHARMANcharman@fr.ibm.com

http://users.polytech.unice.fr/~charman/

Last update: 21-04-2014

Summary

• Testing periods and evolution of testing models

• Software testing timeline

Testing Periods

• Until 1956: Debugging-oriented period

• 1957–1978: Demonstration-oriented period

• 1979–1982: Destruction-oriented period

• 1983–1987: Evaluation-oriented period

• 1988– …: Prevention-oriented period

Debugging-oriented Period( … - 1956)

• No consensus between testing and debugging:– For some people, testing is a sub-activity of

debugging– For others, debugging and testing are the same

activity– For still others, debugging and testing are two

distinguished activities

• The main focus is on the hardware not on the software

Demonstration-oriented Period (1957 - 1978)

• People realize bugs can have a negative impact

• Charles Baker makes a clear distinction from debugging and testing:– Debugging: Make sure the program runs– Testing: Make sure the program solves the problem

• Testers should write tests having in mind they have to be successful

Destruction-oriented Period(1979 – 1982)

• In 1979, G.J. Myers writes his famous book:

Destruction-oriented Period(1979 – 1982)

• Myers defined testing as “the process of executing a program with the intent of finding errors”

• Tester should write tests having in mind they have to fail

• Myers: Showing a program has bugs is more efficient than showing a program has no bugs.

• Google Testing Blog: “If it ain't broke, you're not trying hard enough”

Evaluation-oriented Period(1983 – 1987)

• Guideline for Lifecycle Validation, Verification and Testing of Computer Software

• Methodology that integrates analysis, review and test activities to provide product evaluation during the software life cycle

• A tester can start working on the software life cycle from the very beginning

• A tester can review and gives his feedback on the specifications of the features to be implemented

• For instance a tester can detect that:– User needs won’t be fully satisfied with the feature– A feature has some holes and thus cannot be correctly tested in

some particular cases– etc.

Prevention-oriented Period(1988 – …)

• Quality metrics: • Code coverage• Complexity• etc.

• Rule coding• Comments in the code• etc.

• Continuous Inspection

Prevention-oriented Period(1988 – …)

• Quality metrics: • Code coverage• Complexity• etc.

• Rule coding• Comments in the code• etc.

• Continuous Inspection

Major Testing ModelsPrimary Goal

Phase

Models

Demonstration To make sure the program solves the problem

Destruction To detect implementation faults

Life Cycle

Models

Evaluation To detect requirements, design and implementation faults

Prevention To prevent requirements, design and implementation faults

Software Testing Timeline

Some Companies and Organizations

• 1896 IBM founded• 1958 First software test team • 1973 Compuware founded • 1981 Rational Software founded• 1989 Mercury Interactive founded • 1991 Pure Software founded • 2001 StickyMinds.com launched• 2002 ITSQB founded

Some Tools and Applications

• 1988 First defect tracking tool (DDTS)• 1989 First version of LoadRunner • 1991 Start of the newsgroup comp.software.testing • 1995 First release of Winrunner • 1998 Bugzilla released • 2002 IBM Rational Functional Tester • 2002 JIRA 1.0 released• 2005 Hudson/Jenkins released

Some Models and Standards

• 1970 Waterfall model• 1983 IEEE 829• 1986 V-model• 1993 SCRUM• 1996 Extreme Programming• 2001 Agile Manifesto• 2002 Test Driven Development

References

• Baker C Review of D.D. McCracken’s “Digital Computer Programming”” Mathematical Tables and Other Aids to Computation 1 I, 60. (Oct. 1957). 298-305

• Gelperin, D.; B. Hetzel (1988). "The Growth of Software Testing". CACM 31 (6)• Guideline for Lifecycle Validation, Verification and Testing of Computer Software.

Processing Standards, National Bureau of Standards, June, 1983. • Myers, Glenford J. (1979). The Art of Software Testing. John Wiley and Sons.

Some interesting links

• http://www.stickyminds.com/• http://www.testingreferences.com/• http://googletesting.blogspot.com/• http://idtus.com/img/UsefulAutomatedTestingMetrics.pdf