RANJAN BHAMBROO
Experimental Investigation of Effectiveness of Logical Expressions Using t-way
Testing
Introduction
Experimental Evaluation of logical expressions.
Same idea but continued on new path.
T – way testing instead of pair-wise testing.
New tool for generating logical expressions.
New tool from NIST to construct t-way test sets.
T-way Testing
T-way
Pair – wise = 2-way
T-way where t=2,3,4,5,6.
T- way more effective than pairwise.
But test cases not enough need more test cases.
T-way Testing
Example:For 15 variables total number of
combinations = 215
215 = 32768 combinations
T-way values Test Cases
2-way 10
3-way 24
4-way 58
5-way 131
6-way 296
Tools Used
Boolean Expression Generator – ECU( Developed by Ranjan Bhambroo)
Advanced Combinatorial Testing System(ACTS) for constructing t-way combinatorial test sets – NIST
Fault Evaluator – ECU
Boolean Expression Generator(New Tool)
Generates Logical Expressions Automatically.Simple and user friendly interface. Different Modes to generate Expressions: a)Random Number of variables. b)Fixed Number of variables. c)Interval of variablesCan Control complexity of expressions.Allows user to save expressions in a text file.
Experimental Results Of T-way Testing
Fault Evaluator
Boolean Expression Generator
Advanced Combinatorial
Testing System
Logical
Expressions
T-way
Tes
t
Sets
Effectiveness
Experimental Results Of T-way Testing
Note Using Pair Wise Test Cases
10 expressions/set VNF ORF VRF (ENF) ASF
All Fault Types (weighted avg.)
Set1 36.25 45.58 24.96 60.27 74.51 30.34
Set2 24.67 39.57 17.9 46.55 54.84 22.32
Set3 28.57 43.18 19.54 45.78 75 24.54
Set4 28.06 46.99 21.69 55.95 62.79 26.96
Set5 30.67 39.86 20.38 62.07 60.98 25.3
Average 29.64443.03
620.89
4 54.124 65.624 25.892
Std. Deviation 3.823 2.974 2.376 6.801 7.909 2.678
Experimental Results Of T-way Testing
Note Using 3-Wise Test Cases10 expressions/set VNF ORF VRF (ENF) ASF
All Fault Types (weighted avg.)
Set1 40.88 48.03 27.61 63.93 80 33.27
Set2 26.39 42.54 20.62 48.21 72.41 25.22
Set3 28.97 45.19 22.04 53.23 71.88 26.88
Set4 32.24 52.11 26.02 59.7 76.47 31.52
Set5 36.22 53.85 29.12 68.75 86.67 34.47
Average 32.9448.34
425.08
258.76
477.48
6 30.272
Std. Deviation 5.157 4.201 3.248 7.339 5.457 3.611
Experimental Results Of T-way Testing
Note Using 4-Wise Test Cases10 expressions/set VNF ORF VRF (ENF) ASF
All Fault Types (weighted avg.)
Set1 43.07 53.54 36.11 63.93 95 40.79
Set2 33.33 51.49 27.32 55.36 68.97 31.96
Set3 39.31 54.81 30.16 58.06 93.75 35.49
Set4 34.87 52.82 30.46 62.69 88.24 35.55
Set5 50.39 66.67 41.66 79.17 96.67 47.07
Average40.19
455.86
6 33.142 63.842 88.526 38.172
Std. Deviation 6.14 5.508 5.125 8.266 10.179 5.267
Experimental Results Of T-way Testing
Note Using 5-Wise Test Cases10 expressions/set VNF ORF VRF (ENF) ASF
All Fault Types (weighted avg.)
Set1 47.45 55.91 39.25 67.21 97.5 43.97
Set2 38.19 59.7 33.84 62.5 89.66 38.77
Set3 39.73 57.04 31.57 58.06 96.88 36.69
Set4 42.76 69.86 36.35 64.18 94.12 41.55
Set5 50.39 70.94 45.05 81.25 96.67 50.11
Average43.70
4 62.69 37.212 66.64 94.966 42.218
Std. Deviation 4.601 6.423 4.68 7.881 2.893 4.653
Experimental Results Of T-way Testing
Note Using 6-Wise Test Cases10 expressions/set VNF ORF VRF (ENF) ASF
All Fault Types (weighted avg.)
Set1 48.18 55.91 42.4 67.21 97.5 46.44
Set2 41.67 60.45 39.38 62.6 100 43.56
Set3 42.07 60.74 36.79 59.68 96.88 41.43
Set4 40.79 60.56 37.29 65.67 100 42.31
Set5 53.54 70.94 48.73 81.25 100 53.29
Average 45.25 61.7240.91
8 67.282 98.876 45.406
Std. Deviation 4.902 4.953 4.377 7.449 1.391 4.29
Experimental Results Of T-way Testing
2-way 3-way
Experimental Results Of T-way Testing
4-way 5-way
Experimental Results Of T-way Testing
THANK YOU