24
Mutation Testing Presented by Sharath Kumar Garlapati Vinesh Thummala.

Presented by Sharath Kumar Garlapati Vinesh Thummala

Embed Size (px)

Citation preview

Page 1: Presented by Sharath Kumar Garlapati Vinesh Thummala

Mutation Testing

Presented bySharath Kumar Garlapati

Vinesh Thummala.

Page 2: Presented by Sharath Kumar Garlapati Vinesh Thummala

What is Mutation testing

History of Mutation

Traditional Process of Mutation testing

Kinds of Mutations

Mutation Operators

An Example for Mutation Testing

Pros and Cons

Conclusion

Learning Objectives

Page 3: Presented by Sharath Kumar Garlapati Vinesh Thummala

Mutation Testing is a testing technique that focuses on

measuring the adequacy of test cases

Mutation Testing is NOT a testing strategy like path or data-

flow testing. It does not outline test data selection criteria.

What is Mutation testing?

Page 4: Presented by Sharath Kumar Garlapati Vinesh Thummala

Proposed by Richard Lipton as a student in 1971.

Developed and published by DeMillo, Lipton and Sayward in

late 70’s.

Implemented as a testing tool by Timothy Budd in 1980 from

Yale University.

History of Mutation

Page 5: Presented by Sharath Kumar Garlapati Vinesh Thummala

Mutation Process

[2]

Page 6: Presented by Sharath Kumar Garlapati Vinesh Thummala

The process, given program P and test suite T, is as follows:

Systematically apply mutations to program P to obtain a

sequence,P1, P2,... Pn of mutants of P. Each mutant is derived

by applying a single mutation operation to P.

Run the test suite T on each of the mutants T is said to kill

mutant Pj, if it detects an error.

If k out of n mutants killed, the adequacy of T is measured by

the quotient k/n implies T is mutation adequate, if k = n.

Mutation Process

Page 7: Presented by Sharath Kumar Garlapati Vinesh Thummala

Value Mutation - Involves changing the values of constants or

parameters(by adding or subtracting values etc).

Decision Mutation- Involves in modifying conditions to

reflect potential slips(replacing a > by a < in a comparison).

Statement Mutation - Deleting or Swapping the lines of codes.

Kinds of Mutations

Page 8: Presented by Sharath Kumar Garlapati Vinesh Thummala

Statement deletion

Boolean expression changes (true and false)

Arithmetic operation changes (+ and *, - and /)

Boolean operator changes (> and >=, == and <=)

Declared in the same scope variables changes (variable types

should be the same)

Mutation Operators

Page 9: Presented by Sharath Kumar Garlapati Vinesh Thummala

Polymorphic

Method Overloading

Method Overriding/Hiding in Inheritance

Field Variable Hiding in Inheritance

Information Hiding (Access Control)

Static/Dynamic States of Objects

Exception Handling

Class level mutation operators

Page 10: Presented by Sharath Kumar Garlapati Vinesh Thummala

class T { … }interface K { … }class S extends T implements K { … }class U extends S { … }

The original code:S s = new S();

Mutants:a)T s = new S();b)K s = new S();c)S s = new T(); d)S s = new U();

Polymorphic mutations

Page 11: Presented by Sharath Kumar Garlapati Vinesh Thummala

public LogMessage(int level, String logKey, Object[]inserts){…}

public LogMessage(int level, String logKey, Object insert) {…}

Mutant:public LogMessage(String logKey, int level, Object[]inserts{…}

Method overloading mutations

Page 12: Presented by Sharath Kumar Garlapati Vinesh Thummala

public class Animal { public void speak(){..} public void bite(){..}}public class Cat extends Animal { public void speak() {..} public void bite(){..}}Mutant:public class Cat extends Animal { //public void speak() {..} public void bite(){..}

Method Overriding/Hiding in Inheritance

Page 13: Presented by Sharath Kumar Garlapati Vinesh Thummala

public class Animal {public String kind;public int legCount;

}public class Cat extends Animal {

public String kind;public int legCount;

}

Mutant:public class Cat extends Animal {

public String kind;//public int legCount;

}

Field Variable Hiding in Inheritance

Page 14: Presented by Sharath Kumar Garlapati Vinesh Thummala

The original code:protected Address address;

Mutants:

a)public Address address;

b)private Address address;

c)Address address; //default (no access modifier defined)

Information Hiding (Access Control)

Page 15: Presented by Sharath Kumar Garlapati Vinesh Thummala

The original code:public static int VALUE = 100;private String s;

Mutants: a) public int VALUE = 100; //static is removed.b) private static String s; //static is added.

Static/Dynamic States of Objects

Page 16: Presented by Sharath Kumar Garlapati Vinesh Thummala

The original code:String formatMsg(LogMessage msg){ …

try { …} catch(MissingResourceException mre){…}…

}Mutant:String formatMsg(LogMessage msg)throws MissingResourceException {… //try-catch block removed}

Exception Handling

Page 17: Presented by Sharath Kumar Garlapati Vinesh Thummala

Compares the internal states of the mutant and original

program immediately after execution of the mutated portion of

the program.

Requires much less computing power to ensure that the test

suite satisfies weak mutation testing.

Weak Mutation

Page 18: Presented by Sharath Kumar Garlapati Vinesh Thummala

When strong mutation testing is performed, the values

produced by mutated code and changes (or not) the output of

the module or subroutine.

Requires much more computing power.

Strong Mutation

Page 19: Presented by Sharath Kumar Garlapati Vinesh Thummala

List of tools available –

Ninja Turtles- .NET mutation testing tool

Mutagenesis- PHP mutation testing framework

Heckle- Ruby Mutation Testing Tool

Jester- Mutation Testing Tool for Java

Automation of Mutation Testing

Page 20: Presented by Sharath Kumar Garlapati Vinesh Thummala

It is a powerful approach to attain high testing coverage of the

source program.

This testing is capable of comprehensively testing the mutant

program.

Mutation testing brings a good level of error detection to the

software developer.

Customers are benefited from this testing by getting a more

reliable and stable system.

Advantages

Page 21: Presented by Sharath Kumar Garlapati Vinesh Thummala

Costly and Time consuming.

Many test cases are required.

Requires a lot of testing before a dependable data is obtained.

Needs an automated tool to reduce testing time.

Not an applicable method for black box testing.

Very complicated to use without an automated tool.

DisAdvantages

Page 22: Presented by Sharath Kumar Garlapati Vinesh Thummala

Not just a testing method, more of an analytical method. 

An effective program for improving the quality of testing

software but its widespread use is prevented by the difficulties

encountered in using them.

It would be very helpful to the developers of complex

software, if testing methods could be improved

Conclusion

Page 23: Presented by Sharath Kumar Garlapati Vinesh Thummala

[1] Anguswamy,R., Frakes,W., A Comparison of Database Fault Detection Capabilities Using

Mutation Testing, Proceedings of ESEM,12,(Sept,2012,NY,USA),323-326.

[2] Praphamontripong,U., Offutt,J., Applying Mutation Testing to Web Applications, Proceeding

of ICSTW ,10 ,(Apr,2010, Paris,

[3] Schuler,D., Zeller,A., Efficient Mutation Testing for Java, Proceedings of ESEC/FSE, 06

(Aug. 2009., NY, USA),297-298.

[4] Madeyski, L., Nguyen, Q., Problems of Mutation Testing and Higher Order Mutation Testing ,

Proceedings of ICCSAMA, 12,(2014, Switzerland),157-172.

[5] Khurshid, S., Marinov, D., Zhang, L., Zhang,Lu., Regression Mutation Testing, Proceeding

of ISSTA ,12,(July.2012, MN, USA),331-341.

Reference

Page 24: Presented by Sharath Kumar Garlapati Vinesh Thummala

Thank You