Mining Software Repositories to Assist Developers and Support Managers Ahmed E. Hassan SoftWare...

Mining Software RepositoriesMining Software Repositoriesto Assist Developersto Assist Developers

and Support Managers and Support Managers

Ahmed E. Hassan Ahmed E. Hassan

SoftWare Architecture Group (SWAG)SoftWare Architecture Group (SWAG)

University Of WaterlooUniversity Of Waterloo

ahmed@alumni.uwaterloo.caahmed@alumni.uwaterloo.ca

2

Software Repositories

• Track the evolution of a software project: – source control systems store changes to the code

– defect tracking systems follow the resolution of defects

– archived personnel communications record rationale for decisions throughout the life of a project

• Used primarily for historical record supporting activities such as: checking bug status, or retrieving old code

3

Usage of Software Repositories Beyond Record Keeping

• Prior Research:– SEL - NASA [Basilli, et. al]:

• Fault analysis (Fortran 84)

– 5ESS - Lucent [Eick, Graves, Mockus, Perry, et. al]:• code decay, fault and effort predictions, visualizations, process

– EMERALD - Nortel [Hudepohl, Khoshgoftaar, et. al]– Telephony - Nokia [Gall et. al]:

• reveal hidden dependencies

– Open Source [Chen et. al]: • index code for searching

• Personal Industrial Experience• Open Source Phenomena: easy access to rich repositories

for large projects

4

Survey on Usage of Source Control Repositories in Industry

• Developers:– More likely to record the change rationale than list

implementation alternatives or limitations– Use change msgs to understand legacy code and fix bugs

• Senior developers use messages to:– Track the progress of the project– Monitor the quality of the code (specific developers)– Ensure code ownership boundaries

• Change messages in open source projects are as descriptive as ones in industrial systems

CVS Mailings

Bugzilla

CVS Mailings

Bugzilla

HistoricalInformation

Extraction

Bug Prediction

Resource Allocation

Software Understanding

Change Propagation

CVS Mailings

Bugzilla

HistoricalInformation

Extraction

Bug Prediction

Resource Allocation

Software Understanding

Change Propagation

8

Extracting Information from Source Control Repositories

• CVS tracks changes at the line level, instead an evolutionary extractor tracks:– Addition, removal, or modification of code entities

(functions, variables & macros):• function_1 is added/removed/modified

– Addition and removal of dependencies:• function_1 no longer calls function_2

• Attaches additional CVS change information: developer name, co-changed entities, change reason

9

Complexity of Evolutionary Extraction

main() { int a; /*call help*/ helpInfo(); }

helpInfo() { errorString!} main() { int a; /*call help*/ helpInfo(); }

helpInfo() { int b;} main() { int a; /*call help*/ helpInfo(); }

V1:Undefined func.(Link Error)

V2:Syntax error

V3:Valid code

10

Evolutionary Extractors: Challenges and Complexity

• Unstable & Evolving Code

• Scalability and Robustness– Large amount of data/snapshots– Legacy systems are hard to parse

• Accuracy– Entities with similar names throughout time

Assisting Software Developers

12

Understanding the Software Architecture

File System

MemoryManager

NetworkInterface

ProcessScheduler

Inter-ProcessCommunication

Legend: subsystem depends on

File System

MemoryManager

NetworkInterface

ProcessScheduler

Inter-ProcessCommunication

Legend: subsystem depends on unexpecteddependency

Conceptual(proposed)

Concrete (reality)

Why? Why? Who?Who?

When? When? Where?Where?

13

Dependency Investigation Questions (W4 Approach)

• Which low-level code entity is responsible for a dependency?– Network (SendData) Scheduler (PrintToLog)

• Who added/removed the dependency?– Junior vs. senior/experienced developer

• When was the dependency modified?– Late night / Just before release

• Why was the dependency added/removed?– The rationale!

14

Source StickyNotes

• Static dependencies give a static view of the system – not enough detail!

• Need to extend static dependencies, but how?– Ask developers to fill StickyNotes for each

change– Use software repositories to build these notes

automatically

15

Case Study – NetBSD(VM Subsystem)

HardwareTrans.

Kernel FaultHandler

Pager

FileSystemVirtual Addr.

Maint.VM Policy

Subsystem

Depend

DivergenceHardware

Trans.

Kernel FaultHandler

Pager

FileSystemVirtual Addr.

Maint.VM Policy

Convergence

Subsystem

Why? Why? Who?Who?

When? When? Where?Where?

Conceptual(proposed)

Concrete (reality)

16

Unexpected Dependencies

• Eight unexpected dependencies• All except two dependencies existed since the first day:

– Virtual Address Maintenance Pager

– Pager Hardware Translations

Which? vm_map_entry_create (in src/sys/vm/Attic/vm_map.c) depends on pager_map (in /src/sys/uvm/uvm_pager.c)

Who? cgd

When? 1993/04/09 15:54:59 Revision 1.2 of src/sys/vm/Attic/vm_map.c

Why?

from sean eric fagan: it seems to keep the vm system from deadlocking the system when it runs out of swap + physical memory. prevents the system from giving the last page(s) to anything but the referenced "processes" (especially important is the pager process, which should never have to wait for a free page).

17

Understanding Software Using StickyNotes

• Traditional dependency graphs and program understanding models usually do not use historical information

• In many open source projects, CVS comments are used for:– Communicating new features– Explaining design decisions– Narrating the progress of a project

Supporting Managers

19

Code Development Process

• Central part in making software – Code changes to:– Implement/Enhance features– Repair faults

• Acts as an early warning:– Does a complex code development process produce complex

software?

• Can we:– measure its complexity?– monitor the evolution of its complexity?– predict problems (delayed releases or faults)?

20

Development Process Chaos

A chaotic code development process negatively affects its outcome, the software system, such as the occurrence of faults

Domain

Requirements

Team(size/structure)

SchedulePressure

Code/DesignDevelopment

Process

“Complexity is the business we are in and complexity is what limits us”. Fred Brooks, The Mythical Man-Month

21

Complexity of Development Process

• “To implement feature A we modified one file”

• “To implement feature B we modified too many files – I don’t even remember half of them!”

• Too much information to recallDecay of grasp of what is going in a complex system

22

• Measures the amount of information in a message• Shannon Entropy for a random prob distrib. P

pk = 1:– If all pk = 1/n (equal prob.) maximal entropy

– If exists pi = 1 minimal entropy

• Min. number of bits – Best compression

Information Theory

Hn(P) = - pk log2(pk)

23

0.5

0.3

0.1

0.1A

D

C

B

Time

File

period(for e.g. week)

Info Theory in Dev Process

24

Evolution of Entropy

Time

Entropy

0.5

1

1.5

2

A

D

C

B

period 1 period2 period 3 period 4

0.5

0.3

0.1

0.1A

D

C

B

Time

File

period(for e.g. week)

25

Adaptive System Sizing

• System size changes over time • Standardized Shannon Entropy H:

• Many files are rarely modified (eg. util or libs):– A large n reduces the entropy– Use an active working set instead of n = # of files

H(P) = (log2 n)-1 * Hn(P) = - (log2 n)-1 * pk log2(pk)

26

Postgres – Development Chaos

0.7

0.75

0.8

0.85

0.9

0.95

1

320 330 340 350 360 370 380 390 400

Time (months since 1970)

Trad. Entropy

Adaptive Entropy

Release Dates

6.0 6.3 6.4 6.5 7.0 7.1 7.26.21.09 6.1

“This release marks a major step in the development team's mastery of the source code we inherited from Berkeley. You will see we are now easily adding major features, thanks to the increasing size and experience of our world-wide development team” 6.5 release notes

27

Case Study 2: KDE

0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

330 340 350 360 370 380 390 400

Time (months since 1970)

En

tro

py

Trad Entropy

Adaptive Entropy

KDE 2.0 was also a very ambitious project, and the nature of the changes delayed the project for a whole year”, Miguel de Icaza, founder of GNOME and Ximian

28

Mathematical Validation Details

• Studied repositories of six open source software systems (OpenBSD, FreeBSD, NetBSD, Postgres, KDE, KOffice)

• Built statistical regression models using the 2nd and 3rd years:– Faults

– Changes

– Entropy metric

data to predict faults in 4th and 5th years

29

Results of Mathematical Validation

• Prior faults better fault predictor than prior changes (~50% improvement in prediction error)

• Entropy metric better fault predictor than:– Changes (~13-40% improvement)– Faults (~15-38% improvement)

• Results statistically significant at (95%) using a paired T-test

30

Development Chaos Summary

• A new perspective on the complexity of software – focus on process instead of code

• Entropy metrics used to study the evolution of complexity and predict faults

• Results verified using 6 open source projects

31

Conclusion

• Software repositories contain a wealth of valuable information about the evolution of projects

• Presented approaches and techniques to:– Recover data from source control repositories

– Assist developers in understanding code structure

– Support managers in predicting the occurrence of faults

• MSR likely to take a central role in supporting software development practices and research

33

Thesis Contributions

• Evolutionary Software Extractors (C-REX)• For Developers:

– Source StickyNotes– Development Replay Approach

• For Managers:– Top Ten List– Software Development Chaos

• All contributions were validated through:– case studies using open source systems, and– a survey of software practitioners

34

Research Overview

• Develop approaches and techniques to :– Part I : Extract Information from Repositories– Part II : Assist Developers in

• Understanding the current code structure

• Changing code and propagating changes

– Part III: Support Managers in• Allocating limited testing resources

• Predicting faults

35

Part II: Assisting Developers

• Source StickyNotes:– Assist in architecture understanding

– Attach historical information to the dependency graph

• Development Replay:– Reenacts historical changes using proposed tools and

strategies to assess their benefits

– Permits us to perform empirical based estimation of the benefits of tools and strategies, e.g. change propagation

36

Part III: Supporting Managers

• Top Ten List: – Assists managers in allocating limited resources

• Development Process Chaos: – Develop a complexity metric based on the

process followed by the developers to produce the code instead of on the code or the requirements

37

38

Research Hypothesis

“Software repositories contain a wealth of valuable information about the evolution of a software project. By mining such historical information, we can develop techniques and approaches to support software developers and managers in their endeavors to build and maintain complex software systems”