Upload
sandra-gibson
View
230
Download
4
Embed Size (px)
Citation preview
Mutation Testing
Presented bySharath Kumar Garlapati
Vinesh Thummala.
What is Mutation testing
History of Mutation
Traditional Process of Mutation testing
Kinds of Mutations
Mutation Operators
An Example for Mutation Testing
Pros and Cons
Conclusion
Learning Objectives
Mutation Testing is a testing technique that focuses on
measuring the adequacy of test cases
Mutation Testing is NOT a testing strategy like path or data-
flow testing. It does not outline test data selection criteria.
What is Mutation testing?
Proposed by Richard Lipton as a student in 1971.
Developed and published by DeMillo, Lipton and Sayward in
late 70’s.
Implemented as a testing tool by Timothy Budd in 1980 from
Yale University.
History of Mutation
Mutation Process
[2]
The process, given program P and test suite T, is as follows:
Systematically apply mutations to program P to obtain a
sequence,P1, P2,... Pn of mutants of P. Each mutant is derived
by applying a single mutation operation to P.
Run the test suite T on each of the mutants T is said to kill
mutant Pj, if it detects an error.
If k out of n mutants killed, the adequacy of T is measured by
the quotient k/n implies T is mutation adequate, if k = n.
Mutation Process
Value Mutation - Involves changing the values of constants or
parameters(by adding or subtracting values etc).
Decision Mutation- Involves in modifying conditions to
reflect potential slips(replacing a > by a < in a comparison).
Statement Mutation - Deleting or Swapping the lines of codes.
Kinds of Mutations
Statement deletion
Boolean expression changes (true and false)
Arithmetic operation changes (+ and *, - and /)
Boolean operator changes (> and >=, == and <=)
Declared in the same scope variables changes (variable types
should be the same)
Mutation Operators
Polymorphic
Method Overloading
Method Overriding/Hiding in Inheritance
Field Variable Hiding in Inheritance
Information Hiding (Access Control)
Static/Dynamic States of Objects
Exception Handling
Class level mutation operators
class T { … }interface K { … }class S extends T implements K { … }class U extends S { … }
The original code:S s = new S();
Mutants:a)T s = new S();b)K s = new S();c)S s = new T(); d)S s = new U();
Polymorphic mutations
public LogMessage(int level, String logKey, Object[]inserts){…}
public LogMessage(int level, String logKey, Object insert) {…}
Mutant:public LogMessage(String logKey, int level, Object[]inserts{…}
Method overloading mutations
public class Animal { public void speak(){..} public void bite(){..}}public class Cat extends Animal { public void speak() {..} public void bite(){..}}Mutant:public class Cat extends Animal { //public void speak() {..} public void bite(){..}
Method Overriding/Hiding in Inheritance
public class Animal {public String kind;public int legCount;
}public class Cat extends Animal {
public String kind;public int legCount;
}
Mutant:public class Cat extends Animal {
public String kind;//public int legCount;
}
Field Variable Hiding in Inheritance
The original code:protected Address address;
Mutants:
a)public Address address;
b)private Address address;
c)Address address; //default (no access modifier defined)
Information Hiding (Access Control)
The original code:public static int VALUE = 100;private String s;
Mutants: a) public int VALUE = 100; //static is removed.b) private static String s; //static is added.
Static/Dynamic States of Objects
The original code:String formatMsg(LogMessage msg){ …
try { …} catch(MissingResourceException mre){…}…
}Mutant:String formatMsg(LogMessage msg)throws MissingResourceException {… //try-catch block removed}
Exception Handling
Compares the internal states of the mutant and original
program immediately after execution of the mutated portion of
the program.
Requires much less computing power to ensure that the test
suite satisfies weak mutation testing.
Weak Mutation
When strong mutation testing is performed, the values
produced by mutated code and changes (or not) the output of
the module or subroutine.
Requires much more computing power.
Strong Mutation
List of tools available –
Ninja Turtles- .NET mutation testing tool
Mutagenesis- PHP mutation testing framework
Heckle- Ruby Mutation Testing Tool
Jester- Mutation Testing Tool for Java
Automation of Mutation Testing
It is a powerful approach to attain high testing coverage of the
source program.
This testing is capable of comprehensively testing the mutant
program.
Mutation testing brings a good level of error detection to the
software developer.
Customers are benefited from this testing by getting a more
reliable and stable system.
Advantages
Costly and Time consuming.
Many test cases are required.
Requires a lot of testing before a dependable data is obtained.
Needs an automated tool to reduce testing time.
Not an applicable method for black box testing.
Very complicated to use without an automated tool.
DisAdvantages
Not just a testing method, more of an analytical method.
An effective program for improving the quality of testing
software but its widespread use is prevented by the difficulties
encountered in using them.
It would be very helpful to the developers of complex
software, if testing methods could be improved
Conclusion
[1] Anguswamy,R., Frakes,W., A Comparison of Database Fault Detection Capabilities Using
Mutation Testing, Proceedings of ESEM,12,(Sept,2012,NY,USA),323-326.
[2] Praphamontripong,U., Offutt,J., Applying Mutation Testing to Web Applications, Proceeding
of ICSTW ,10 ,(Apr,2010, Paris,
[3] Schuler,D., Zeller,A., Efficient Mutation Testing for Java, Proceedings of ESEC/FSE, 06
(Aug. 2009., NY, USA),297-298.
[4] Madeyski, L., Nguyen, Q., Problems of Mutation Testing and Higher Order Mutation Testing ,
Proceedings of ICCSAMA, 12,(2014, Switzerland),157-172.
[5] Khurshid, S., Marinov, D., Zhang, L., Zhang,Lu., Regression Mutation Testing, Proceeding
of ISSTA ,12,(July.2012, MN, USA),331-341.
Reference
Thank You