Moving from Waterfall to Agile

Kalpana Sureshchandra, Jagadish Shrinivasavadhani Wipro Technologies

kalpana.suresh@wipro.com, jagadish.shri@wipro.com

Abstract

In a crisis ridden business environment, customers have become very averse to surprises. Business windows have become smaller; there is a heightened need for shorter development cycles and higher visibility. All this is translating into more and more customers specifically asking for agile. Service organizations such as Wipro Technologies need to adopt lean and agile methodologies to support the transition. As agile coaches, the biggest challenge we face is in transitioning the mindset of the team from that of a waterfall model to an agile thought pattern. Our experience in converting a waterfall team to agile is shared in this report. 1. Introduction:

We are part of a central team of agile coaches. At

Wipro, we use a blend of Extreme Programming and Scrum methodologies which can be tailored depending on project and customer needs. As a part of deploying agile, we conduct training and hand-hold/coach/mentor the teams.

Initially we took the approach of mandating all agile practices upfront to the project teams. This did not work – though the benefits of practices like short iterations, pair programming, test driven development, continuous integration etc. are high, the team members developed cold feet when they really had to practice them.

We now realize that in our first coaching projects, we tried to command and control the team rather than making them self-driven. We applied agile to the very process of deploying Agile. To overcome initial apprehensions, we suggested that teams start with a few “must have” agile practices specific to the project and then scale up on rigor as required. This approach worked. Teams started off with a tailored version of Agile. In due course, we converted the teams from “sparsely agile” to “almost agile” – our experience in

bringing in this transformation is detailed out in this paper.

2. Project Background

This case pertains to a team from the embedded domain. The customer was a large player in the area of computer peripherals. The project involved enhancing the existing systems in C/C++, with a planned production release every 3-4 months. The 8 member team followed a waterfall cycle in the previous releases. As a first step, we conducted a retrospective of their current development process and elicited the following problems:

• The team was overworking during the 4-5 weeks preceding the release.

• Many times the release date had to be postponed due to delays in development and testing.

• Too many defects were raised by the customer during acceptance testing.

After conducting training on agile, rather than prescribing the agile practices, we asked the team to choose the required agile practices that would solve the above stated problems. After deliberations, the team decided on the following:

• The release would have two 6-week iterations. The team believed that the iteration length could not be reduced any further. They felt that having a demo mid-way in the release would help them get early feedback.

• The project manager insisted that the complete plan for 6 weeks had to be drawn out upfront and that there would be no reduction in documents.

• The team refused to adopt test driven development, pair programming and continuous integration. They opined that automation would not have any significant benefits.

• The concept of doing analysis, design, coding and testing simultaneously was not

Agile 2008 Conference

978-0-7695-3321-6/08 $25.00 © 2008 IEEE

DOI 10.1109/Agile.2008.49

97

comprehensible to the team. They wanted to follow a mini waterfall cycle. We subtly pointed out that they need to start testing early to solve their stated problems. On our suggestion, the team agreed to have weekly builds from the second week of the iteration.

• The project manager explained that she would estimate the effort for the tasks as she was the most experienced person and had a good understanding of the business and technical challenges.

In short, the team had picked up only time-boxing and daily standup meetings from the set of agile practices. With our urging, they added intermediate weekly retrospectives. 3. Transforming the waterfall project

The project was hardly agile, but we wanted the

team to experiment and learn at their pace within the confines of expected deliverables.

3.1. First week (Analysis and Design):

The team had completed the analysis and design

tasks planned for the week. During the intermediate retrospective, the team pointed out that standup meetings were time consuming and not adding much value. On further analysis we found that:

• Except the project manager, all others were focused on their stories/tasks. So, during the standup meetings, stories/tasks were discussed in detail, which was one of the reasons for extended standup meetings. We suggested having a story board, which would be helpful in tracking stories and in disseminating the information to the entire team. Though the team agreed, the project manager did not think that this would help. She agreed to it because we volunteered to invest the initial efforts setting it up and the team would not need much time to update it. The board had the following columns: Not Started, Analysis, Design, Coding + Unit-Testing, System Integration Testing, User Acceptance Testing, and Released.

• To further shorten the daily standups one person in rotation was selected as the coordinator, who would focus the team to talk about what was done yesterday, plans for today and problems encountered. The coordinator would also prevent the team from discussing any other details.

• It was also noted that some issues highlighted during the scrum meetings were getting

overlooked and not tracked to closure. The team decided to list all open issues in a centrally located white board for full visibility of the entire team.

At the end of first week, the team understood the nuances of conducting effective stand up meetings. 3.2. Second week (Design and Coding + Unit

testing): The team could not fulfill their second week plan.

During coding, the team had to update the design document as some parts were either missed out or not documented correctly. This consumed some unplanned effort. The team was supposed to start on the weekly builds from the second week, but the integrated build never happened. We focused on the slippage from the plan and the weekly build during the intermediate retrospective.

• Since some effort was spent on documentation, we prompted the team to look into it in detail and check if everything in the current design template was needed. We questioned if elaborate documentation was needed as the team was communicating much more than it used to. It was proposed by the team to capture only the design decisions (that might be needed for future iterations or during the maintenance phase) in the document. This was later discussed with the customer and agreed upon.

• None of the developers had checked in their code for the weekly build because they were not sure if the code was working. As agile coaches, we pitched in to explain how test driven development could have helped them in ensuring that the developed code actually met the requirements. The team agreed to write tests first; this would definitely not be the complete set, but they would make a start and add more tests as they proceeded with coding.

• The project manager pointed out that the teams were moving the stories frequently between design and coding. This made the current status unclear. We quickly explained that an agile project would not have distinct phases for design and coding. The team understood the concept of parallel “design” and “coding” by practical experience. These phases were tracked together on the storyboard thereby reducing the confusion on status. The big win was the team’s willingness to practice test driven development. The project manager was

98

getting used to a democratic team. Though she felt that the team had “suggested” the actions and she had approved them, the move away from command-and-control had begun.

3.3. Third week (Coding + Unit testing):

The team was back on track with respect to plan.

They had stretched a little bit to get things under control. After a few failed builds, a successful build was finally done on Thursday. During the retrospective:

• Self- discipline was emphasized. A senior person drew up a checklist on things to do before checking in the code and circulated it as hardcopy. It was also suggested to use this as the mouse-pad.

• The project manager pointed out that once there was a build, the most logical thing to do was to test. The team agreed to start testing from the fourth week.

• The project manager was upset that the entire plan had to be changed. We pointed out that the team could use story boards instead of the current Microsoft Project Plan. We explained that the story board would be visible to all and any changes could be made easily. She welcomed this suggestion.

The project manager was worried that if the team was totally self-organized, there would not be any need for her. We explained how her role would change from controlling the team to removing roadblocks which impede the team's progress. Her manager, who is a Certified Scrum Master, helped us in encouraging her to change her management style. 3.4. Fourth week (Coding + Unit testing and

System integration testing):

The team was definitely behind schedule at the end of the fourth week. Testing identified a number of defects and the team spent more time than planned fixing them. The story board had one more enhancement – the delayed stories would have additional indicators (in the form of colored magnetic pins). Highlights of the retrospective:

• Since the build frequency was weekly (on Wednesdays) the team did not know if defects were fixed correctly. They had to wait until next week to ascertain the fix. The team realized that they needed more frequent builds for effectively closing the defects and the build frequency was increased to thrice a week (Monday, Wednesday and Friday). The project manager felt that the increase in build

frequency would be a burden to the build manager. We reminded her on her new management style and she halfheartedly agreed to the team’s decision.

• At this juncture, we asked if it would be better if the team estimated the remaining effort rather than the project manager. The project manager got the idea of what we were alluding to. She agreed that the estimates would be more realistic if it came from the team and there would be more ownership and enthusiasm to complete the iteration on time. She welcomed the decision and most of the team members drew up their own estimates for the remaining effort.

• The team realized most of the defects were because (a) the analysis was not complete or (b) the dependencies were not fully understood. As agile coaches, we asked if pairing would have helped – the project manager and the team agreed that pairing during analysis phase would have reduced the defects.

The team was starting to understand the benefits of agile. The team felt the need to increase build frequency (without us forcing them). They were moving closer to a self-organizing team where pairing was seen as a useful tool rather than as overhead. 3.5. Fifth week (Coding + Unit testing and

System integration testing):

The team was working hard to meet the iteration goals. There was visibly more individual and collective ownership within the team. By this week, most of the development work was done and system integration testing was the focus. The build frequency had increased to daily. The project manager was beginning to realize the intangible benefits of a self-organizing team. Realizations during the weekly retrospective were:

• The build manager was spending a lot of effort on the builds.

• While the testing effort increased, the effectiveness of testing was decreasing as the same tests were repeated after every fix.

• The manual code inspection exposed parts of code that had not followed coding standards (due to work pressures) or could have been written better.

The project manager pointed out that automation would be essential if we were to continue the daily build-test setup. This was a big win for us – as the team previously said they did not want automation.

99

One team member pointed out that had we done daily reviews, they could have identified defects earlier in the life cycle. With just one more week to go in the current iteration, the team decided that these would be explored during the next iteration.

3.6. Sixth week (System integration testing):

The team almost made it. Most of the planned stories were completed – barring a few known defects/issues. The demo was given to the customer. There were a few defects and some suggestions, which were recorded and would be addressed in the next iteration.

While the team was exhausted, there was a glimmer of hope as they had a firm grip on what they would do different next time. 3.7. Learning from the iteration:

The sixth week retrospective focused on the full 6 weeks of the first iteration. The following decisions were taken:

• The first problem pointed out was late integration. For the next iteration, the team decided to have nightly builds.

• Automation was identified as another area for improvement. Builds would be automated using CruiseControl; Builds would be followed by automated testing using C++Test.

• Efforts would be made to strengthen test driven development. Tests would be written upfront and would be updated as design/coding progressed.

• Since most of the defects were due to insufficient analysis, the team decided to follow partial pair programming. Two stories would be owned by a pair. The analysis would definitely be done in pairs; the remaining tasks may or may not be paired. The pair owns the collective responsibility of getting the stories done.

• After analysis, the team would meet again to discuss their work in detail so that dependencies/omissions (if any) could be identified early. Coding work would not begin until the stories were discussed in detail with the entire team.

• During this meeting, the pair would need to come up with the estimated effort for their stories. The estimates would be discussed in detail with all members and adjustments (if any) would be made.

The team definitely moved closer to agile. However the iteration length was still 6 weeks.

3.8. Iteration Two:

The team went on to complete iteration two,

implementing all the improvement actions identified during the earlier retrospective.

• Builds were automated; C++Test was used effectively. There were a few broken builds – but they were rectified on top priority.

• Many people did not check in their code until the second or third week. For the next release, the team decided to measure when each person was participating in the build – the last person to check in would have to take the entire team out for lunch.

• Pairing continued beyond the analysis. After a few hours of coding, the pairs would meet again to verify the code and test.

• The project manager was pleased with the ownership and energy levels of the team. With such a motivated team, she could focus only on removing the stumbling blocks and help the team in achieving their goals.

The output of at the end of second iteration had better quality. And for the first time the team had no defects during acceptance testing. There were a few suggestions from the customer, which took a week to resolve. Customer satisfaction was at an all time high. The number of times the team had to stretch beyond planned hours was significantly less and the team morale was higher. 3.9. Future releases:

Before planning for the next release, the project

manager suggested that the team should try 5-week iterations instead of 6. The team felt otherwise – they wanted 4-week iterations so that the feedback from the customer could be sought earlier. Though the project manager was skeptical, she realized that in the last three months the team’s commitment was higher for decisions made by them.

Participation in early builds, number of successful builds and the automation rate continued to increase. Team started measuring code complexity metrics, which acted as the trigger for refactoring. After a couple of releases the team started following 3-week iterations.

100

4. What we learned

As agile coaches, our key takeaways were: • Follow an agile approach to move from

waterfall to agile – teams may not be ready to start with all the practices.

• Even when the team is starting with just a few practices, handholding/mentoring from an agile coach is needed. A seemingly simple practice like standup meeting also needs guidance in the initial stages.

• Agile coaches should wear the hat of a mentor/counselor and guide the team towards agile. A dogmatic attitude on agile adoption is counter productive.

• For most teams, practices like test driven development, pair programming and continuous integration are difficult to implement. Allow the team to start small and gradually increase the rigor depending on their project needs.

• During the first few weeks of agile adoption, involvement of the agile coach needs to be high: conducting group and individual mentoring. Getting the team to be self-disciplined needs constant attention and needs to be handled diplomatically.

• Steer the team to choose the agile practices depending on their current problems. For example, rather than stressing on practices, a practical demonstration of the problems they face due to lack of these helps in building the appreciation for the same.

• Choose the measurements carefully. What is measured influences the way people work. For example, when we started measuring just the number of successful builds, developers would not check-in the code early as they wanted to be thorough. Measuring the number of days from the start of iteration to the first successful build encouraged the team to start building early.

• Changing the mindset of project managers takes some time. They like to hear the experience from their peers. During our initial stages of agile adoption, we recommended the Certified Scrum Masters program to our project managers. Apart from hearing from experts, this gave them an opportunity to meet and interact with other

agile managers. After having executed many projects within the organization, we now encourage new project managers to talk to other managers who have already practiced agile.

5. Conclusion

Moving from waterfall to agile cannot be done

overnight or in a single step. It takes time for people to unlearn old traditional practices and move towards agile. Coaches need to be patient, positive, and persistent while bringing in this change. Agile practices call for a change in mindset. If some people are inflexible and refuse to change their ways, it is best not to have these people in an agile project. For the majority, the conversion needs to be done gradually and diligently.

In this paper we have summarized the agile adoption of one project. This is not the only way to do it. Even within our organization other projects have followed their own methods – some starting with 2-week iterations or the engineering disciplines upfront. Our suggestion to all agile coaches is to respond to changing needs of the team rather than following a preconceived plan in adopting agile.

6. References [1] Mike Cohn, Agile Estimation and Planning [2] Ian Wilson, http://www.scrumalliance.org/articles/79-

agile-its-the-business-stupid [3] Anthony Heath, Why switch a failing waterfall project

over to agile?; http://www.scrumalliance.org/articles/73-why-switch-a-failing-waterfall-project-over-to-agile

[4] John Hill, It's All in the Delivery;

http://www.scrumalliance.org/articles/18-its-all-in-the-delivery

[5] Matt Roadnight & Toby Barber, Using Scrum to deliver

a phase of a waterfall project : Experiences and Warnings; http://www.scrumalliance.org/resources/281

[6] Mike Cohn; Patterns of Agile Adoption;

http://www.agilejournal.com/articles/articles/patterns-of-agile-adoption.html

101