Upload
idalee
View
29
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Empirical Validation of OO Metrics in Two Different Iterative Software Processes. Mohammad Alshayeb Information and Computer Science Department King Fahd University of Petroleum and Minerals http://www.alshayeb.com. - PowerPoint PPT Presentation
Citation preview
Empirical Validation of OO Metrics in Two Different
Iterative Software Processes
Mohammad Alshayeb
Information and Computer Science Department
King Fahd University of Petroleum and Mineralshttp://www.alshayeb.com
Alshayeb, M. and Wei Li, “An Empirical Validation of Object-Oriented Metrics in Two Iterative Processes,” IEEE Transactions
on Software Engineering, Vol. 29, No. 11, November 2003.
4 January 2004 2
Agenda
Research Objective. Introduction.Research Data.Research Hypotheses.Validation Studies.Conclusion.Future Work.
4 January 2004 3
Objective
Validate the Predictive Capability of Object-Oriented (OO) Metrics in Two Iterative Software Processes.
4 January 2004 4
Measurement Acceptance
No Wide Acceptance of Software Measurements Because: Management personnel rarely know how
to analyze, interpret or use the software metrics.
Various ways of analyzing software metrics.
Lack of measurement validation.
4 January 2004 5
Measurement Validation
Validation Assures That Measurements Assess What They Are Supposed to Measure.
Types of Validation: Theoretical: the measure is a proper numerical
characterization of a claimed attribute. Empirical: establishes the accuracy of a prediction
system by comparing the model performance with known data in a given environment.
4 January 2004 6
Software Processes
The Waterfall ModelPrototype ModelRapid Application Development ModelThe Incremental ModelThe Spiral ModelComponent Assembly ModelThe Iterative Development Process ModelExtreme Programming
4 January 2004 7
The Iterative Development Process
Domain Analysis
Release Iteration
Prototype Highest Risk
Risk Assessment
Software Architecture
Requirements definition
Test Suite & Environment Development
Integrate with Previous Iteration
Customer Requirements
Iteration
Luckey et al. , “Iterative Development Process with Proposed Applications,” 1992
4 January 2004 8
Agile Process: Extreme Programming (XP) “A lightweight, efficient, low-risk,
flexible, predictable, scientific, and fun way to develop software”
-Beck,
XP Applicability: Projects with vague or changing requirements. Small groups of programmers. The need for customer input.
4 January 2004 9
XP Is Different
Continuity of Feedback Throughout Its Short Cycles.
Incremental Planning.Flexibility in Scheduling on the Basis of
Business Needs.Dependency on Unit Tests Written by
Both Developers and Customers.Pair Programming.
4 January 2004 10
XP Breakdown
Customer System
Priority Organization of user stories
Story 1 Story 2 Story 3
Division of system into user stories
Iteration 1 Iteration 2 Iteration 3
Iterations assigned for the development of each story
Story 2 Story 3 Story 1
Release 2 Release 3Task 1 Task 3
Test Case 2Test Case 1 Test Case 3
Release 1Working, tested system with story 2
integrated
Passed, integrate with the system
Task 2
4 January 2004 11
XP and Other Software Processes
Analysis
Design
Implementation
Testing
Waterfall Long Cycled Iterative Process Short Cycled Iterative Process XP
Beck, “Embracing changes with Extreme programming,” 1999
4 January 2004 12
Frameworks
“A Framework is an OO class hierarchy plus a built-in model which defines how the objects derived from the hierarchy interact with one another”
–Lewis et al,
4 January 2004 13
OO Metrics Suites
C&K suite Weighted Methods per
Class (WMC) Depth of Inheritance Tree
(DIT) Number Of Children
(NOC) Coupling Between Objects
(COB) Response For a Class
(RFC) Lack of Cohesion of
Methods (LCOM)
Li Suite Number of Ancestor Classes
(NAC) Number of Local Methods
(NLM) Class Method Complexity
(CMC) Number of Descendent
Classes (NDC) Coupling Through Abstract
data types (CTA) Coupling Through Message
passing (CTM)
4 January 2004 14
Research Data
Iterative Software Process
Process: Extreme Programming (XP) Process: the Framework Iterative
Data: Two Java Applications Data: JDK Releases
4 January 2004 15
Research Data
System 1
Story 1
Story 2
Story 3
Story 4
Story 5
Story 6
JDK
JDK 1.0
JDK 1.1
JDK 1.2
JDK 1.3
JDK 1.4
System 2
Story 1
Story 2
Story 3
Story 4
Story 5
Story 6
Story 7
4 January 2004 16
Data Collection
The tasks planned for the current iteration cycle. The tasks come from the user stories, which are supplied by the customer.
A description of the progress or failure of progress towards completing the tasks.
The time spent on each task measured by hours. A description of the problems encountered during
implementing a task and the attempted solutions. A description of the changes made on the
system, the reasons for the changes, and the affected classes.
4 January 2004 17
Data Analysis
Tools support Webgain’s Quality AnalyzerMetamata
Metrics Object-Oriented metrics collection. Collects 12 metrics.
Krakatau Project Manager for Java Collecting development/maintenance effort
measured by lines of code.
4 January 2004 18
Metrics Validation
Criteria Li and Henry
Basili et al. Briand et al.
EL Emam et al.
Briand and Wüst
Suite of metrics
CK metrics CK metrics
Cohesion and
Coupling
C&K, Lorenz
and Kidd
Size, Cohesion
and Coupling
CK& and Li metrics
CK& and Li metrics
Type of products
OO dialect of Ada
C++ Ada Java C++ Java Java
Dependent variable
Number of changes in
components
Fault occurrence
in C++ classes
Fault occurrence
in Ada packages
Fault-proneness of a class
effort Design Changes (Lines
added, changed, and deleted), in a
class.
Maintenance effort, refactoring
effort
Statistical technique
Least-square
regression
Logistic regression
Logistic regression
Logistic regression
Poisson regression
and regression
trees
Least-square regression
Least-square regression
Long-Cycled and Short-Cycled
Iterative Processes
Short-Cycled Iterative Processes
Alshayeb and Li
Process?
4 January 2004 19
Multiple Regression
Investigate the Relationship Between a Dependent Variable and Multiple Independent Variables
y = x +xx The x’s are the independent (predictor or regressor)
variables. y is the dependent (response) variable. —the statistical error—is the difference between the
observed value y and the line (x +xx )
regression coefficient. R2: The variance percent in the dependent variable
based on the knowledge of the independents.
4 January 2004 20
Empirical Validation of Object-Oriented MetricsEmpirically Validate the Predictive
Capability of OO Metrics in the Long-Cycled (Framework) and Short-Cycled (Agile) Iterative Software Processes Investigate the relationship between metrics and
development/maintenance efforts in two forms of iterative processes.
Types of Changes: Lines added (LA), Lines changed (LC), and Lines
deleted (LD). Refactoring and Error-Fix efforts.
4 January 2004 21
Example of Lines Added, Deleted and Changed// Example.java version 1
public class Example{
public static void main(String args[]){
System.out.println("Welcome to java");
System.out.println("Hello");
System.out.println("World");
}
}
// Example.java version 2
public class Example{
public static void main(String args[]){
System.out.println("Hello");
System.out.println("Java");
System.out.println("new World");
}
}
Deleted
Added
Changed
4 January 2004 22
Research Hypotheses
Hypothesis 1: Using OO metrics, we can predict design changes indicated by added, changed, and deleted source lines of code in classes from one iteration (release) to the next in the long-cycled framework iterative process.
Hypothesis 2: Using OO metrics, we can predict added, changed, and deleted source lines of code in classes from one iteration to the next in the short-cycled XP iterative process.
4 January 2004 23
Research Hypotheses
Hypothesis 3: Using OO metrics, we can predict error-fix maintenance effort (measured in man-hours) in classes from one iteration to the next in the short-cycled XP iterative process.
Hypothesis 4: Using OO metrics, we can predict refactoring effort (measured in man-hours) in classes from one iteration to the next in the short-cycled XP iterative process.
4 January 2004 24
Dependant and Independent Variables -Hypotheses 1 and 2Dependant Variables Are:
LA, LC, and LD. Independent Variables
WMC, DIT, LCOM, NLM, CTA, and CTM metrics.
4 January 2004 25
Research Results -Hypothesis 1
Hypothesis 1
Dependent
Variables JDK1.0-1.1 JDK1.1-1.2 JDK1.2-1.3 JDK1.3-1.4
R2 % R2 % R2 % R2 %
LA 42.6 10.6 27.8 32.7
LC 56.0 18.6 24.9 13.6
LD 58.0 11.0 12.9 7.30
4 January 2004 26
Research Results -Hypothesis 1
Object-Oriented Metrics Cannot Predict Design Changes Indicated by Added, Changed, and Deleted, Source Lines of Code in Classes From One Iteration (Release) to the Next in the Long-cycled Framework Iterative Process.
4 January 2004 27
Research Results -Hypothesis 2
Hypothesis 2
Dependent
Variables Story 5-6
R2 %
LA 95.6 LC 97.9 LD 88.5
4 January 2004 28
Research Results -Hypothesis 2
Hypothesis 2
Dependent
Variables Story 4-5 Story 6-7
R2 % R2 %
LA 97.7 54.2 LC 98.4 72.9 LD 71 51.5
4 January 2004 29
Research Results -Hypothesis 2
Object-Oriented Metrics May Predict Design Changes Indicated by Added, Changed, and Deleted Source Lines of Code in Classes From One Iteration to the Next in the Short-cycled XP Iterative Process Only When System Design Accumulates to Certain Mass.
4 January 2004 30
Dependant and Independent Variables -Hypotheses 3 and 4Dependant Variables Are:
Refactoring and Error-Fix efforts. Independent Variables
WMC, DIT, LCOM, NLM, CTA, and CTM metrics.
4 January 2004 31
Research Results –Hypotheses 3,4
Hypotheses 3 and 4
System 1 Refactoring Error Fix
R2 % R2 %
Story 5 76.5 88.3
Story 6 72.9 69.4
4 January 2004 32
Research Results –Hypotheses 3,4
Hypotheses 3 and 4
System 2 Refactoring Error Fix
R2 % R2 %
Story 4 61 44.5
Story 5 66.0 74.4
Story 6 14.3 10.0
Story 7 91.2 90
4 January 2004 33
Research Results –Hypotheses 3,4
Object-oriented Metrics Can Predict the Refactoring Effort and Error-fix Maintenance Effort in the XP-like Process When the System Design Accumulates to Certain Mass.
4 January 2004 34
Conclusion
The Predictive Capability of OO Metrics: OO metrics are reasonably effective in predicting
design changes in the short-cycled XP-like process, but they are largely ineffective in the long-cycled framework evolution process.
OO metrics' predictive capability is limited to the design and implementation changes during the development iterations, not the long-term evolution of an established system in different releases.
OO metrics are effective in predicting error-fix and refactoring efforts in the short-cycled agile process.
4 January 2004 35
Conclusion
The Development Software Process is Related Directly to the Predictive and Project-Progress Indicative Capabilities of OO Metrics: It is thus essential to clearly identify the
software process when validating OO metrics.
4 January 2004 36
Future Work
Studies Can Target OO Metrics Capabilities in Different Software Processes.
Validation Studies More Diverse Data Sets Are Needed to Test the Prediction Accuracy.
Automate the Steps Followed in This Study and Build a Tool to Gather, to Analyze, and to Use Metrics and the Statistical Models in a CASE Tool.
4 January 2004 37
Questions