Embed Size (px)
Citation preview
Stakeholder Perceptions of the Adoption of Continuous Integration – A Case StudyEero Laukkanen (Aalto University),Maria Paasivaara (Aalto University),Teemu Arvonen (Ericsson)
What is Continuous Integration?
“Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible.”
- Martin Fowler
18.04.20232
Why This Study?
Continuous integration practices in the industry vary a lot and are not all really continuous, but more infrequent.“Automated Software Integration Flows in Industry: A Multiple-case Study” by Ståhl and Bosch, 2014.
Adopting continuous integration has social challenges, such as adopting the mindset and setting clear goals.“Challenges When Adopting Continuous Integration: A Case Study” by Debbiche et al., 2014.
Idea: Could we study individuals to understand these challenges better?
18.04.20233
Questions
1. What actions were done to adopt continuous integration?
2. What challenges were faced during the adoption?
3. How was the adoption perceived by different stakeholders?
4. How would different stakeholders proceed with the adoption?
18.04.20234
Ericsson Organization• Platform + Services• Early market lifecycle• 11 teams, >100 people• Releases every 8 weeks
18.04.20235
Large-Scale & Distributed Complex Platform
Team
Development site
Operations site
Business systemcomponent
Network systemcomponent
Site CSite B
Site A Site D
Research Approach - Interviews
18.04.20236
Site A Site B Site C Total
Team member 12 4 3 19
Product owner 2 1 1 4
Other 1 2 1 4
Total 15 7 5 27
• What does CI mean to you?• Why do you practice CI?• Experiences of adoption• Challenges of adoption• What should be done next?
What actions were done to adopt continuous integration?
18.04.20237
2011 2012 2013 2014 2015
Platform acquiredStarted agile
Started building CIon Site A
CI roadshows
Interviews
CI experts planted
Watchdogs
What challenges were faced during the adoption?Lack of Time
Difficult Components
Unstable Tests
Slow Tests
Insufficient Testing Environments
Disagreement on Tools
Global Distribution of the Organization
18.04.20238
“I don’t think we have the capacity in the both in the development organization to do this at the same time as we do everything else we have committed.”
What challenges were faced during the adoption?Lack of Time
Difficult Components
Unstable Tests
Slow Tests
Insufficient Testing Environments
Disagreement on Tools
Global Distribution of the Organization
18.04.20239
“[A component] has not been designed to be tested or version controlled.”
What challenges were faced during the adoption?Lack of Time
Difficult Components
Unstable Tests
Slow Tests
Insufficient Testing Environments
Disagreement on Tools
Global Distribution of the Organization
18.04.202310
“No, [the tests] are not working. It's, maybe 30 or something like that, that fails every time and no-one cares because we know that they fail.”
What challenges were faced during the adoption?Lack of Time
Difficult Components
Unstable Tests
Slow Tests
Insufficient Testing Environments
Disagreement on Tools
Global Distribution of the Organization
18.04.202311
“It's many hours now, the nightly build. It's running several hours and... I'm not sure how many more test cases we can actually add to that before it starts taking too long.”
What challenges were faced during the adoption?Lack of Time
Difficult Components
Unstable Tests
Slow Tests
Insufficient Testing Environments
Disagreement on Tools
Global Distribution of the Organization
18.04.202312
“There is a problem that, some of our, our test environments, like, I say staging for example, it doesn't have full connectivity at the moment.”
What challenges were faced during the adoption?Lack of Time
Difficult Components
Unstable Tests
Slow Tests
Insufficient Testing Environments
Disagreement on Tools
Global Distribution of the Organization
18.04.202313
“It's quite easy to write [test cases], but I experienced that the test framework was very buggy. So it was difficult to do the test cases.”
What challenges were faced during the adoption?Lack of Time
Difficult Components
Unstable Tests
Slow Tests
Insufficient Testing Environments
Disagreement on Tools
Global Distribution of the Organization
18.04.202314
“So, we use video conferences, it's very hard to say what [the participants from Site A] say, they turn to each other, talk to each other within that little group, and eventually, we're, hey, we're here, could we say something.”
How was the adoption perceived by different stakeholders?Most were positive on Site A
Some were critical on Site B
How CI was adopted
Enforcement of automated test cases
No clear vision or strategy
Some were neutral on all sites
Not using product CI
18.04.202315
“We have found quite many faults from the system, and also if some modification changes something unintentionally, then we notice that quite well with the cases we have at the moment.” Site CSite B
Site A Site D
How was the adoption perceived by different stakeholders?Most were positive on Site A
Some were critical on Site B
How CI was adopted
Enforcement of automated test cases
No clear vision or strategy
Some were neutral on all sites
Not using product CI
18.04.202316
“I think it could have been done in a much more efficient way, if it had been driven in a more open-minded manner and if it had been, a communication rather than an idea which was imposed.”
Site CSite B
Site A Site D
How was the adoption perceived by different stakeholders?Most were positive on Site A
Some were critical on Site B
How CI was adopted
Enforcement of automated test cases
No clear vision or strategy
Some were neutral on all sites
Not using product CI
18.04.202317
“I don't really see it. I just see meeting notes, I see focus area discussion in our employee meetings that we should focus on it, but I don't really see the results.”Site CSite B
Site A Site D
How would different stakeholders proceed with the adoption?Change the Architecture
Proper Testing Environments
Better Communication
Testing Strategy
Decide Who Creates Test Cases
Need Support for Creating Test Cases
18.04.202318
“We should clean the vague signaling between the systems, such as databases. We should standardize the signaling between the systems.”
How would different stakeholders proceed with the adoption?Change the Architecture
Proper Testing Environments
Better Communication
Testing Strategy
Decide Who Creates Test Cases
Need Support for Creating Test Cases
18.04.202319
“I think the next step is to have of course this stage environment, that would be the first thing.“
How would different stakeholders proceed with the adoption?Change the Architecture
Proper Testing Environments
Better Communication
Testing Strategy
Decide Who Creates Test Cases
Need Support for Creating Test Cases
18.04.202320
“Looking at CI for its own sake, [people at Site A] need to open up and be much more perceptive of the overall system and the requirements the system have on ways of working.”
How would different stakeholders proceed with the adoption?Change the Architecture
Proper Testing Environments
Better Communication
Testing Strategy
Decide Who Creates Test Cases
Need Support for Creating Test Cases
18.04.202321
“When it comes to the verification I think we should really, really think through which nodes, which parts of the system we want to make automated test cases for.”
How would different stakeholders proceed with the adoption?Change the Architecture
Proper Testing Environments
Better Communication
Testing Strategy
Decide Who Creates Test Cases
Need Support for Creating Test Cases
18.04.202322
“We should probably also have dedicated resources, that are experts with the test framework.”
How would different stakeholders proceed with the adoption?Change the Architecture
Proper Testing Environments
Better Communication
Testing Strategy
Decide Who Creates Test Cases
Need Support for Creating Test Cases
18.04.202323
“The competence needs to be spread more. There has been some trainings, which are good, but if you don't start the work immediately then you forget it.”
Implications for Practice
Architecture has a significant effect on CI adoption
Provide time to overcome the initial learning phase
Invest in distributed communication and ensure that every site has the competences needed
18.04.202324
Thank you!
Questions?
