Studying the Evolution of Build Systems

Studying the Evolution of Build

SystemsShane McIntosh

Queen’s University

What is the build system?

2

Build systems help practitioners

3

OK Testers

Developers

Managers

Build systems require 12%

of a developer’stime (on average)

4

Kumfert, G., and Epperly, T.Software in the DOE: The

Hidden Overhead of the “Build”

Address Bar Search Bar

5

4 months beforethe issue was fixed!

Research hypothesisBuild system maintenance imposes a

significant degree of overhead on software development

6

Studied projects

PLplot7

>30 MLOC!

Thesis Overview

1.0 1.1 2.01.High level

evolution analysis

2. Fine-grainedevolution analysis

3. Maintenance Overhead analysis

8

Acceptance Ratio:16/51 = 31%

1.0 2.0

Acceptance Ratio: 62/441 = 14%

1.0 2.0

High level evolution

1.Code analysis: Do the static size and complexity of source code and build system evolve similarly?

2.Runtime analysis: Does the build-time complexity evolve?

91.0 1.1 2.0

Our approach

10

Code analysis(Static)

1.Do the static size and complexity of source code and build system evolve similarly?

2.Does the build-time complexity evolve?

111.0 1.1 2.0

12

ANTLRAbstraction

Module Restructuring

Module Restructuring

Source & Build Change Together!

Runtime analysis (Dynamic)

1.Do the static size and complexity of source code and build system evolve similarly?


131.0 1.1 2.0

Libs builtfrom source

Build length increases over time

14

High level evolution results1.Do the static size and complexity of

source code and build system evolve similarly?


15

Yes, but events may impact build and

source differently.Yes, length tends to

grow as projects age.

Release granularity is too

coarse!1 2 31.1 1.21.1.1 1.1.2 1.1.3

Fixed bug #1234

Added feature #1212

Issue #4444: restructured

module16

Fine-grained evolution

1.How many files does a typical build system consist of?

2.How much does a typical build system churn?

3.How large are typical build system changes?

171.0 2.0

Revision tagging

1.1.1 1.1.2

Fixed bug #1234 Added feature #1212

Issue #4444: Restructured

module18

Revision tagging

Fixed bug #1234

19

Size of the build system




201.0 2.0

Build accounts for 9% of files (median)

21

Build system rate of change




221.0 2.0

High churn may result in post-release bugs!

Normalized churn is on par with source code

23

Size of build changes




241.0 2.0

Build changes add and delete 3-4

linesSource changes add and delete 6-

11 lines25

26

Fine-grained evolution1.How many files does a typical build

system consist of?



9% of filesOn par with

sourceAdds and deletes 3-4

lines

How do projects cope with build maintenance?

27

Build maintenance overhead

1.How often are build changes required to complete development tasks?

2.How do projects distribute build maintenance work?

281.0 2.0

Logical coupling

29

LC(Source ⇒ Bld)

= 2 ÷ 4 = 50% 1.0 2.0

Logical coupling at revision level is

low!

30

Source ⇒ Build coupling well below 12% estimation!

Work item grouping

1.1.1 1.1.2

Fixed bug #1234 Added feature #1212

Issue #4444: Restructured

module31

Work item grouping

1.1.1 1.1.2

Fixed bug #1234Oops, forgot somecode for bug

#1234

Work item grouping

Bug #1234

33

Fixed bug #1234Oops, forgot somecode for bug

#1234

Task-centric analysis



341.0 2.0

Work item coupling to the

build is high

35

Mozilla’s Source ⇒ Build coupling is very high!

Developer-centricanalysis



361.0 2.0

If you produce source code, do you produce

build code?

37

Dispersed Ownership:

Build maintenance is

distributed amongst most team members

38

Concentrated Ownership:

A small team isresponsible for most

of the build maintenance

Build maintenance overhead1.How often are build changes

required to complete development tasks?


Up to 27% require build changes

Dispersed or concentrated

Potential bias Studiedprojects

Allprojects

40

41

Questions?