View
227
Download
0
Category
Preview:
Citation preview
8/3/2019 PTesting vs Ctesting Touseef
1/38
Understandablity of Concrete versus Parametric Test Cases
By
Touseef Tahir, Ali Jafar, Mohan Maharajan
Lahore
8/3/2019 PTesting vs Ctesting Touseef
2/38
hIntroduction to Unit Testing
Concrete Testing AND Parametric Testing
h
The research QuestionshHypothesis
hPlanning and design of Experiment
hExperiment
hAnalysis and discussion
hConclusion
8/3/2019 PTesting vs Ctesting Touseef
3/38
hA test case is a description of a specific interactionthat a tester will have in order to test a singlebehavior of the software. Test cases are very
similar to use cases, in that they are step-by-stepnarratives which define a specific interactionbetween the user and the software.
hTest case is a triplet [I, S, O] where
I is input dataS is state of system at which data will be input
O is the expected output
hTest suite is set of all test cases
8/3/2019 PTesting vs Ctesting Touseef
4/38
Testing of functional and non functional
behavior of the software programs.
Verifies correctness of the units of program inisolation.
Detect and remove faults earlier from the
specified units.Provides 100% code coverage, checks all the
corner cases and detects error in components
logic
8/3/2019 PTesting vs Ctesting Touseef
5/38
executes for a single test input at an instance.
execute a method of a class under test withfixed arguments and verify that it returns the
expected result or not.
The concrete input values are provided to the
unit to determine its failure or success.
8/3/2019 PTesting vs Ctesting Touseef
6/38
Numerous test inputs that has to be examined
and revised.
Tedious, labor intensive, error-prone, time
consuming, unlikely to cover all possible
behavior and often difficult task.
8/3/2019 PTesting vs Ctesting Touseef
7/38
Does not ensure the high code coverage since
only single examplary input value is provided
as an input.
Increase in effort, cost, time, numerous test
cases, and numerous iterations.
8/3/2019 PTesting vs Ctesting Touseef
8/38
Parameters as an input to the unit of code.
Exercised different code paths for the set of
test inputsDetects more faults than concrete test cases.
High code coverage with small test suite.
Can generate more test instances and executesautomatically.
Less time consuming, and likely to cover all
the possible behavior.
8/3/2019 PTesting vs Ctesting Touseef
9/38
Assist in reusability, evolution, and
maintainability
Hard to measure since depends on cognitive
behavior of human
8/3/2019 PTesting vs Ctesting Touseef
10/38
8/3/2019 PTesting vs Ctesting Touseef
11/38
Documentation of source code
Understanding of Structure and component
Code Complexity
Psychological Complexity
8/3/2019 PTesting vs Ctesting Touseef
12/38
Reading Scalability for general unit test
questions
Closed ended questionnaires
8/3/2019 PTesting vs Ctesting Touseef
13/38
8/3/2019 PTesting vs Ctesting Touseef
14/38
Types Definition
Subjects MSc. Software Engineering
MSc. Computer Sciences
MSc Security engineering
Environment Class Room
Apparatus Document containing Questioners
(Close ended question)
Experimental Design Method Factorial Design
Programming Languages Java, C#
Testing Tools JUnit, NUnit, Pex
Groups JUnit Concrete Testing (6 Subjects)
JUnit Parameterized Testing(6
Subjects)
NUnitConcreter(6 Subjects)
Pex Parameterized Testing (6Subjects)
8/3/2019 PTesting vs Ctesting Touseef
15/38
Variable type Variables
Independent Subjects
Programming languages (Java,C#)
JUnit, NUnit, Pex
Treatment Concrete Testing
Parameterized Testing
Dependent Understandability
Time
Accuracy (Correctness)
Extraneous Participants Gender/Sex, Age
Experiment situation(noise, temperature)
Equipment (fault equipment, calibration)
The time of the experiment
Conduct of experiment
Carry over effect
8/3/2019 PTesting vs Ctesting Touseef
16/38
Understandablity ofConcrete versus
Parametric TestCases
8/3/2019 PTesting vs Ctesting Touseef
17/38
hPurposed Hypothesis H0: Concrete test
cases are difficult to understand than
Parameterized test cases and thus makeoverall software testing difficult.
hAlternative Hypothesis H1: Parameterized
test cases are difficult to understand than
concrete test cases and thus make overall
software testing difficult.
8/3/2019 PTesting vs Ctesting Touseef
18/38
8/3/2019 PTesting vs Ctesting Touseef
19/38
General questions
Questions regarding concrete test cases and
parameterized test cases related to the codeand Test Code.
8/3/2019 PTesting vs Ctesting Touseef
20/38
8/3/2019 PTesting vs Ctesting Touseef
21/38
0
0.5
1
1.5
2
2.5
3
3.5
4
SCALE
8/3/2019 PTesting vs Ctesting Touseef
22/38
Note:
CC=C# Concrete, JP=Java Concrete
Time is measured in Minute, Correct Answer out of 37
8/3/2019 PTesting vs Ctesting Touseef
23/38
Note:
CP=C# Parameterized JP=Java Parameterized
Time is measured in Minute, Correct Answer out of 37
8/3/2019 PTesting vs Ctesting Touseef
24/38
Descriptive Statistics
Hypothesis Testing
Wilcoxon Signed Test
8/3/2019 PTesting vs Ctesting Touseef
25/38
hSummarize the available data
8/3/2019 PTesting vs Ctesting Touseef
26/38
8/3/2019 PTesting vs Ctesting Touseef
27/38
8/3/2019 PTesting vs Ctesting Touseef
28/38
hAccept OR Reject hypothesis
8/3/2019 PTesting vs Ctesting Touseef
29/38
8/3/2019 PTesting vs Ctesting Touseef
30/38
Note: As there is one tie in the data so the number of participants
for this group will be 11 and the table value is 10
8/3/2019 PTesting vs Ctesting Touseef
31/38
8/3/2019 PTesting vs Ctesting Touseef
32/38
hTime
No significant difference
Not practical to conclude on the basis of time
hCorrectness ratio
Significant difference as well as participants
hWe reject null hypothesis
Correctness ratio and general testing questionswe concluded that
hMore general test cases are harder to understand than concrete ones
and thus would lead to tests that are harder to understand.
8/3/2019 PTesting vs Ctesting Touseef
33/38
Understandability of concrete test cases
versus general test cases using mixed
approach.
Using practitioners besides using students.
Using more than two programminglanguages, more subjects and more examples.
8/3/2019 PTesting vs Ctesting Touseef
34/38
8/3/2019 PTesting vs Ctesting Touseef
35/38
8/3/2019 PTesting vs Ctesting Touseef
36/38
h After the early analysis of data it was seen that subjects
have taken different amount of time to solve a specific
example. In some cases the time difference was huge so it
was not fitted for parametric hypothesis testing methods.The data was not in intervals as well so it was decided non
parametric statistics was used on the collected data. The
only factor involved in experiment was unit testing with
two different treatments which were Parameterized test
cases and Concrete test cases. The best suitable method forhypothesis testing was Wilcoxon signed test
8/3/2019 PTesting vs Ctesting Touseef
37/38
8/3/2019 PTesting vs Ctesting Touseef
38/38
Recommended