Quality Assurance and Software Evolution - EmpiricalSE

8/12/2019 Quality Assurance and Software Evolution - EmpiricalSE

1/26

Empirical Software Engineering with Examples

Dr. Cristina Marinescu

Empirical Software Engineering

a sub-domain of software engineering focusing onexperiments on software systems

devise experiments on software, in collecting data fromthese experiments, and elaborate laws and theories fromthis data (wikipedia)

2


Software Quality

?

Defects (Bugs)

3 Dr. Cristina Marinescu

Classication

4

Observe Defects

TRUE FALSE

PredictedTRUE TP FP

Defects FALSE FN TN

precision = TP/(TP + FP)

recall = TP/(TP + FN)


2/26


Some tests...

arithmetic mean

mean(2, 3, 4) = 3mean(0, 3, 6) = 3

standard deviation


Rhttp://www.r-project.org/

6


T. Zimmermann, R. Premraj, and A. Zeller

Predicting defects for EclipseThird International Workshop on Predictor Models in SoftwareEngineering (PROMISE)

IEEE Computer Society, 2007


Predicting Defects for Eclipse

8

Num er of Files Packages

2.0 6729 377

2.1 7888 434

3.0 10593 661


3/26


Used Metrics...

9

methods

FOUTMLOCNBDPARVG

Number of method calls (FAN OUT)Method Lines of CodeNested block depthNumber of parametersMcCabe Cyclomatic Complexity

classesNOFNOMNSFNSM

Number of eldsNumber of methodsNumber of static eldsNumber of static methods

lesACDNOINOTTLOC

Number of anonymous type declarationsNumber of interfacesNumber of classesTotal lines of code

packages NOCU Number of les


Spearman correlations

10

FOUTMLOCNBDPARVG

0.4000.4030.3920.3500.389

0.3190.3220.3200.2830.315

0.5370.5450.3920.5540.546

0.5230.5440.3200.5260.538

NOFNOMNSFNSM

0.2600.3190.1860.202

0.2040.2680.1700.179

0.5070.5020.4590.448

0.4800.4910.4140.371

ACDNOINOTTLOC

0.2580.160(-)0.1600.421

0.1800.129(-)0.1290.333

0.4420.1290.5180.581

0.4140.1100.4700.559

NOCU - - 0.514 0.461


Classication of les

11

Release Testing Precision Recall

2.02.02.13.0

0.6920.4780.613

0.2650.1910.171

2.12.02.13.0

0.6640.6680.717

0.2030.1600.139

3.02.02.13.0

0.5780.5280.675

0.2770.2200.224


Foutse Khomh, Massimiliano Di Penta, Yann-Gal Guhneuc,Giuliano Antoniol

An exploratory study of the impact of antipatterns on class change- andfault-proneness.

Empirical Software Engineering 17(3): 243-275 (2012)

12


4/26

D r. C ri st in a M ar ine sc u 1 3

.

.

40

50

60

70

80

90

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18Releases

% o

f c

l a s s e s

p a r

t i c

i p a

t i n g

i n

a n t i p a t

t e r n s

ArgoUML Eclipse Mylyn Rhino

Fig. 1 Percentages of classes participating in antipatterns in the releases of the four systems


List of AntiPatterns...

14

AntiSingleton

Blob

ClassDataShouldBePrivate

ComplexClass

LargeClass

LazyClass

LongMethod

LongParameterList

MessageChain

RefusedParentBequest

SpaghettiCode

SpeculativeGenerality

SwissArmyKnife


Research Questions...



16


5/26





18


. ;

Dr. Cr is ti na Ma ri nes cu 20

.:

.

.

. :

.

..

.:

.

.

.


6/26


.:

Dr. Cr is ti na Ma ri nes cu 22

Are the clients of awedclasses (also) defect prone?

Authors: Radu & CristinaMarinescu

IEEE International Working Conference on SourceCode Analysis and Manipulation, 2011

W o r k i n g i s h

a r d...


7/26

... e s p e c i a l l y w

h e n r e s o u r c

e s a r e a w e

d

Clients of Flawed Classes

vs.Defects

D o e s a n y c o

r r e l a t i o n e x i

s t i n

E c l i p s e ?

D a t a

C l a s s

F e at u r e E n v y

G o d C l as s B r a

i n C l

a s s

has defects (pre and post release)

hasdesign aws

has provider classeswith design aws

?


8/26

Chi-Square

Odds RatioMann-Whitney

Non-parametric statistical tests awed providers correlates with defects !

aws / awed providers

vs.defects

Odds Ratioon Defects

HASDesign Flaws

NODesign Flaws

HASFlawed Providers

NOFlawed Providers -


HASDesign Flaws

NODesign Flaws

HASFlawed Providers

NOFlawed Providers

0.5 - 0.7 -


9/26


HASDesign Flaws

NODesign Flaws

HAS

Flawed Providers 2 - 3

NOFlawed Providers

0.5 - 0.7 -


HASDesign Flaws

NODesign Flaws

HAS

Flawed Providers 4 - 8 2 - 3NOFlawed Providers

0.5 - 0.7 -


HASDesign Flaws

NODesign Flaws

HASFlawed Providers 4 - 8 2 - 3

NOFlawed Providers

0.5 - 0.7 -

The biggest danger is using awed classes !

Percent ofclasses with... Design Flaws

NODesign Flaws

Flawed Providers

NOFlawed Providers


10/26


NODesign Flaws

Flawed Providers12

NOFlawed Providers

7


NODesign Flaws

Flawed Providers12

42NOFlawed Providers

7 40

Are the Classes that useExceptions Defect Prone?

Author: Cristina Marinescu

12th International Workshop on Principles on Software Evolution7th ERCIM Workshop on Software Evolution

2011

E x c e p t i o n s

h i g h l i g h t e x c

e p t i o n a l


11/26

S h o u l d W E C

A R E M O R E a

b o u t ?

Exceptions

&Defects

D o e s a n y c o

r r e l a t i o n e x i

s t i n

E c l i p s e ?

Class

Location,Name, Type

Class


12/26

Location,Name, Type

Throws,Catches

Class

Location,Name, Type

Throws,Catches

Defects (Pre&Post Release)

Class

Chi-Square &

Odds Ratio

N o n - p a r a m e

t r i c S t a t i s t i c a l

T e s t s

No Exceptions

T h e S t r u c t u r

e o f t h e

C o n t i n g e n c y

T a b l e s.


13/26

Defects NoDefects

No Exceptions

T h e S t r u c t u r

e o f t h e


T a b l e s.

Defects NoDefects

No Exceptions

Exceptions T h e S t r u c t u r

e o f t h e


T a b l e s.

p-value ? 0.05 p-value < 0.05

C o r r e l a t i o n

e x i s t s !


14/26

Defects NoDefects

No Exceptions < >

Exceptions >

Exceptions >

Documents

Quality Assurance and Software Evolution - EmpiricalSE