Build waiting time in continuous integration

Eero Laukkanen Mika Mäntylä

FAIL

developer

CI server

build waiting time

feedback

developer

developer

92% test coverage

?

?

?

? ? ? ? ? ?

?

?

build waiting timeoptimal waiting time

CI context

other contexts

effects

Research questions

1. How build waiting time affects developers in continuous integration practice?

2. What is an optimal waiting time in such practice?

3. What can be learned from other waiting time research to understand the effects of build waiting time?

First literature review

Second literature review

First literature review

Google Scholar & Scopus

“continuous integration” AND (“build time” OR “build length”)

Scopus: 2 articles Scholar: 1 relevant article

Total: 3 articles

Second literature review

Google Scholar

1. unsystematic searches to find waiting time contexts -> three contexts: services, web use, computer use

2. forward snowballing to find latest literature reviews -> found a newer source for computer use

Total: 3 articles

?

FAIL

CI specific cognitive emotional

1. How build waiting time affects developers in continuous integration practice?

FAIL

CI specific

…increases commit size …decreases

commit frequency

…increases build down time

…increases integration effort

Longer build waiting time…

cognitive

…breaks development flow

…delays feedback

Longer build waiting time…

emotional

…decreases developer satisfaction

…decreases team discipline

Longer build waiting time…

?

2 minutes J

2-10 minutes K

10+ minutes L

2. What is an optimal waiting time in such practice?

? ??

? ? ?3. What can be learned from other waiting time research to

understand the effects of build waiting time?

service useweb use computer use

service use

…can be perceived as shorter by modifying

the service environment…decreases

customer satisfaction

Longer service waiting time…

web use

…decreases user satisfaction

…breaks flow of thought

…interferes short-term memory

Longer page waiting time…

…can be perceived as shorter by

providing feedback

computer use

…can be tolerated on

specific tasks

… causes measurable

stress, anxiety

… causes anger, frustration,

annoyance

Longer delays when using computer…

FAIL

2 minutes J

Baker, J., & Cameron, M. (1996). The effects of the service environment on affect and consumer perception of waiting time: an integrative review and research propositions. Journal of the Academy of Marketing Science, 24(4), 338–349.

Brooks, G. (2008). Team Pace – Keeping Build Times Down. In Proceedings of the Agile 2008 (pp. 294–297). Washington, DC, USA: IEEE Computer Society.

Dabrowski, J., & Munson, E. V. (2011). 40 years of searching for the best computer system response time. Interacting with Computers, 23(5), 555–564.

Nah, F. F.-H. (2004). A study on tolerable waiting time: how long are web users willing to wait? Behaviour & Information Technology, 23(3), 153–163.

Rasmusson, J. (2004). Long Build Trouble Shooting Guide. In C. Zannier, H. Erdogmus, & L. Lindstrom (Eds.), Extreme Programming and Agile Methods - XP/Agile Universe 2004 (pp. 13–21). Springer Berlin Heidelberg.

Rogers, R. O. (2004). Scaling continuous integration. Extreme Programming and Agile Processes in Software Engineering, Proceedings, 3092, 68–76.