Historical and Impact Analysis of API Breaking Changes: A Large-Scale Study

Historical and Impact Analysis of API Breaking Changes: A Large-Scale Study

Laerte Xavier, Aline Brito, André Hora, Marco Tulio ValenteFederal University of Minas Gerais, Brazil

SANER 2017

2

API

APIs are constantlychanging!

3

4

5

default Query<R> setHibernateFlushMode(FlushMode flushMode) {

setFlushMode( flushMode );

return this;

}

default org.hibernate.query.Query<R> setHibernateFlushMode(FlushMode

flushMode) {


return (org.hibernate.query.Query) this;

}

hibernate/hibernate-orm

Version 5.2.0

6Version 5.2.0

default org.hibernate.query.Query<R> setHibernateFlushMode(FlushMode

flushMode) {


return (org.hibernate.query.Query) this;

}

7

Breaking Changes

Elements removal

Method signature changes

Non-Breaking Changes

Changes in deprecated elements

Elements addition

Elements deprecationField changes

(Dig and Johnson, ICSM, 2006)

8

Related Work

Hora and Valente, ICSME, 2015

Hora et al., CSMR, 2014

Henkel and Diwan, ICSE, 2005

Raemaekers et al., ICSM, 2012

API Evolution Change Impact

Brito et al., SANER, 2016

McDonnel et al., ICSE, 2013

Trabalhos Relacionados

Hora and Valente, 2015;

Hora et al., 2014;

Henkel and Diwan, 2005.

Raemaekers et al., 2012;

Evolução de APIs Impacto de Mudanças

Brito et al., 2016;

McDonnel et al., 2013.

Open Questions

9

To what extent are clients affected by API breaking changes?



Hora et al., 2014;




Brito et al., 2016;



Open Questions

10

Is this a problem only faced by newer libraries?

11

RQ1: What is the frequency of API breaking changes?

RQ2: How do API breaking changes evolve over time?

RQ3: What is the impact of API breaking changes in client applications?

RQ4: What are the characteristics of libraries with high and low frequency of breaking changes?

12

TOP-317GITHUB JAVA LIBRARIES

13

TOP-3171,792

STARS

GITHUB JAVA LIBRARIES

14

TOP-3171,792

STARS

15RELEASES


15

TOP-3171,792

STARS

15RELEASES

3.4YEARS


16





17

...1 2 NN-1

Design

3 N-2

First Commit

August2016

18

...1 2 NN-1

Design

3 N-2

First Commit

August2016

DIFF

19

501,645LIBRARY

CHANGES

28%BREAKING CHANGES

20

501,645LIBRARY

CHANGES

28%BREAKING CHANGES

21

501,645LIBRARY

CHANGES

28%BREAKING CHANGES

22

Libraries often break backward compatibility!

23





24

...1 2 NN-1

Design

3 N-2

First Commit

August2016

25

...1 2 NN-1

Design

3 N-2

First Commit

August2016

DIFF

DIFF

26

...1 2 NN-1

Design

3 N-2

First Commit

August2016

DIFF

DIFFDIFF

27

...1 2 NN-1

Design

3 N-2

First Commit

August2016

DIFF

DIFFDIFFDIFFDIFFDIFFDIFF

28

1

...2 NN-1

Design

3 N-2

First Commit

August2016

DIFF

29

9,329RELEASES

5YEARS

30

9,329RELEASES YEARS

20%

5

31

Breaking changes frequency increases over time!

32





DIFF

33

NN-1

Design

JAVALI

263,425 Projects

16,386,193Files

http://java.labsoft.dcc.ufmg.br/javali

Boa

34

140,460BREAKING CHANGES

16,291CHANGED TYPES

1,290TYPES WITH

POSSIBLE IMPACT

35

140,460BREAKING CHANGES

16,291CHANGED TYPES

1,290TYPES WITH

POSSIBLE IMPACT

36

Most breaking changes do not have a massive

impact on clients!

37





Design

38

Design

Top (58)

Bottom (58)

0% BC

100% BC

39

Design

Top (58)

Bottom (58)

0% BC

100% BC

(Mann-Whitney U Test & Cliff’s Delta)

40

Bold: p-value < 0.05 (statistically significant different) and d > 0.147 (at least small size effect)

41

Bold: p-value < 0.05 (statistically significant different) and d > 0.147 (at least small size effect)

42

43

Development and social coding measures are associated with API breaking changes!



Hora et al., 2014;




Brito et al., 2016;



Open Questions

44

Is this a problem only faced by newer libraries?

45

Libraries often break backward compatibility

46


Breaking changes frequency increases over time

47



Most breaking changes do not have a massive impact on clients

48



Most breaking changes do not have a massive impact on clients

Development and social coding measures are associated with API breaking changes

49

Would you need tools to measure the impact of breaking changes?

As an API developer...

50

Would you use stability metrics to choose an API?

Would you need tools to measure the impact of breaking changes?

As an API developer...

As an API client...

Historical and Impact Analysis of API Breaking Changes: A Large-Scale Study

Laerte Xavier, Aline Brito, André Hora, Marco Tulio ValenteFederal University of Minas Gerais, Brazil

SANER 2017

Software

Historical and Impact Analysis of API Breaking Changes: A Large-Scale Study