Introducing agile customer-centered development in a legacy software product line

SOFTWARE – PRACTICE AND EXPERIENCESoftw. Pract. Exper. 2011; 41:871–882Published online 12 April 2011 in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/spe.1063

Introducing agile customer-centered development in a legacysoftware product line

Jan Bosch∗,†,‡ and Petra M. Bosch-Sijtsema

Chalmers University of Technology, Gothenburg, Sweden

SUMMARY

The ability to rapidly respond to customer interest and to effectively prioritize development effort has beena long-standing challenge for mass-market software intensive products. This problem is exacerbated in thecontext of software product lines as functionality may easily fall over software asset and organizationalboundaries with consequent losses in efficiency and nimbleness. Some companies facing these problemsin their product line respond with a new development process. In this paper we discuss the developmentswithin a single case study, Intuit’s Quickbooks product line that combined agile software development,design thinking and self-organizing teams in a successful approach, which provided a significant improve-ment in terms of responsiveness and accuracy of building customer value. Copyright q 2011 John Wiley& Sons, Ltd.

Received 31 August 2010; Revised 19 November 2010; Accepted 10 February 2011

KEY WORDS: software product lines; agile software development; compositional software engineering;design thinking

1. SETTING THE CONTEXT

Software product lines (SPLs) present the most successful approach to intra-organizational reuseand many companies have managed to establish or expand their market leadership, often bydisrupting competitors, through the successful adoption of SPL [1, 2]. Although SPLs can beplanned, often the company starts off with a single wildly successful product that then grows intoa family of products. Over time, however, SPLs may become victims of their own success andstart to suffer from a number of challenges, including high coordination cost, slow release cycles,and high system-level error density [3].

The cause of these challenges is what we refer to as integration-oriented software engineering, i.e.treating the entire SPL as one, often large, tightly coupled set of software assets that require virtuallycontinuous coordination between development teams and a sizeable central organization. Startingwith the road mapping and planning stage, through requirements management, development, testingand, finally, integration, teams need to interact, adjust to each other’s interfaces and changes infunctionality behind those interfaces. This approach assumes a slow release cycle, often yearlyor even more slowly, as the cost and unpredictability of the integration phase is so high that theorganization naturally seeks to minimize the pain associated with that stage of development (cf. [4]).

The typical answer to the above is the introduction of an agile development approach [5].Although agile development approaches are very valuable, these are not sufficient in the context

∗Correspondence to: Jan Bosch, Chalmers University of Technology, Gothenburg, Sweden.†E-mail: [email protected]‡The author worked at Intuit as Vice President Engineering Process while the research was performed.

Copyright q 2011 John Wiley & Sons, Ltd.

872 J. BOSCH AND P. M. BOSCH-SIJTSEMA

of large-scale software development in software-intensive product companies [6]. The scale ofdevelopment often requires decoupling mechanisms that further decrease the dependencies betweenteams, such as independent deployment techniques that allow teams to release their code inde-pendently of other teams. In the case of software-intensive product companies, actively involvingcustomers, which may range in the millions, in the prioritization and implementation of function-ality is challenging as any direct customer input is a small sample that may easily be biased whereasany accumulated, averaged customer data is often colored by product management’s assumptionsabout the typical customer.

The topic and contribution of this paper is twofold. First, we present the main issues of SPLand Agile methods for large-scale SPL engineering. Second, we present an example to SPLengineering that encompasses agile, design thinking, and self-organization principles by means ofa company case study.

The paper is organized as follows. In Section 2 we present the theoretical background on whycompositional product line engineering matters. After that we present the research methodologyapplied for collecting data in a single case study. In Section 4 we discuss the case study inwhich this approach has been applied. Section 5 discusses a novel approach to SPL developmentcombining elements from agile, design thinking, and self-organization and presents the specificimplementation of this approach in the case study. The paper is concluded with a discussion andconclusion section.

2. WHY COMPOSITIONAL PRODUCT LINE ENGINEERING MATTERS

The topic addressed in this paper is pervasive and touches many aspects of SPL development. As aconsequence, the problem statement that is fundamental to the paper also has multiple dimensions.Specifically, we focus on three aspects, i.e. SPLs, agile software development, and organizing forlarge-scale software development.

2.1. SPLs

SPLs present the single most successful approach to intra-organizational reuse of software assets[1, 2, 7]. As we discussed in the introduction, companies have established or expanded market lead-ership positions in a variety of industries through the successful introduction of an SPL. However,there are inherent risks associated with successful product lines that may cause the company toturn the initial advantage into a liability. This liability expresses itself [3] in three symptoms:

1. Coordination overhead: Successful SPLs naturally increase the scope of functionality as wellas products over time. This can occur both in an organic fashion, through the introductionof new products, or inorganically through mergers and acquisitions or reorganizations wherepreviously unrelated products are brought into the SPL. With increasing scope of the SPLand the significant increase in the number of people working on it, the cost of coordinatingthe efforts of teams, individuals, product derivation efforts, road mapping, integration ofthe platform, etc. increases exponentially to the point that most staff spends most of itstime coordinating through meetings, email exchange, or other mechanisms and has rapidlydecreasing amounts of time dedicated to adding value to the products and shared assets inthe SPL.

2. Slow release cycles: In case studies presented in earlier researches we described that theshared part of the SPL, i.e. the platform, was integrated and released periodically to theproduct teams that derived new or evolved existing products built on top of the platform[4, 6]. With the increasing size and complexity of the overall SPL as well as the platform, thecost of verifying all functional and quality requirements tends to become very high and as aconsequence, the release frequency decreases and slows to a point that in some companieswas below the ‘speed of the market’. This caused the benefit of the SPL to turn into a liabilityas the SPL caused delays in product releases.

Copyright q 2011 John Wiley & Sons, Ltd. Softw. Pract. Exper. 2011; 41:871–882DOI: 10.1002/spe

AGILE CUSTOMER-CENTERED DEVELOPMENT 873

3. High system-level error density: One of the main reasons for slow release cycles was thatmany errors were only found during the integration stage as the complexity of the platformand the SPL as a whole had reached a point where teams and their architects were unableto predict the implications of their design decisions and extensions on the overall system.Hence, the negative implications were found late in the development cycle and could causetime-consuming rework in the various components. This caused the integration stage tobe long and painful, putting strong pressure on the organization to perform integration asinfrequently as possible, reinforcing the aforementioned problem of slow release cycles.

Although the problems of large, legacy SPLs discussed above are more serious and complexin a product line context, large-scale software engineering in general is suffering from theabove problems among others. In response to this, the last decade has seen an increasingand broad adoption of agile software development. The following section discusses the chal-lenges associated with agile approaches in large-scale, legacy software development for productcompanies.

2.2. Agile software development

Over the last decade, several agile software development process approaches have been developed,including XP [8], lean software development [9], and scrum [10, 11]. In the context of smaller scalesoftware development projects, agile development projects [12] have shown significant success.

Inspired by the agile approaches, especially for web applications and services, software teamsnow focus on small team size, short release cycles, ranging from weeks to several times per day,and experimentation in the market place, i.e. the notion of perpetual beta.

Agile development has been widely documented [5, 8] as working well for small (<10 devel-opers) co-located teams. From the agile software literature it becomes clear that agile teams workmainly co-located, have frequent face-to-face contact, and highly motivated team members whowork in self-organized teams. Techniques, such as pair-wise programming, daily standup meetings,and sprint planning meetings, rely to a large extent on the team members being co-located. Asa consequence, agile development has shown success especially in smaller software developmentprojects. Recently more researchers are studying product lines complemented by agile methods insoftware development [13, 14].

Specifically for large-scale software development in product companies, there are specific chal-lenges in using agile development methods:

• Customer representation: Companies serving mass markets typically have a product manage-ment function to represent the customer internally. However, although product managementmay be able to identify what features are relevant to the customer, providing specific inputabout the implementation of new functionality is often something that product developmentis asked to interpret on its own. Agile methods assume active customer involvement as a keyingredient to make sure that the right functionality is built. In the case of product companies,alternative mechanisms are required to make sure that the customer is sufficiently involved inthe development process. This problem is exacerbated in the context of a SPL, as there aremultiple products, serving different segments of the market, as well as a platform, which isonly indirectly used by customers.

• Architecture evolution: Several authors (e.g. [15]) have studied the relationship between archi-tecture and agile development, but most of this work has focused on the upfront effortsaround architecture. Agile methods focus on the notion of architecture refactoring. Localrefactoring works well independent of scale. However, especially global refactoring whereadding, changing, or removing a design decision has a system-wide effect is very complicatedand goes against the culture that agile teams seek to develop.

• Managing scale: Finally, as an overall concern, the scale of the code base and the associateddevelopment organization requires operating mechanisms that are not covered and may evenbe in conflict with agile development methods. Agile methods are best suited for projects ofmodest to moderate size [5, 16].



2.3. Organizing for large-scale software development

One of the topics that has received more attention in the software engineering research communityin the recent years, is how one organizes large-scale software development where dozens orhundreds of engineers all work on a multi-million line code base.

Most organizations the authors have worked with or for in the past use the notion of componentteams. Using the software architecture of the system as a blueprint, teams of engineers, possiblyextended with an architect, product manager, and/or designer, are responsible for a component,maintain its quality and extend its functionality in response to requests from component teamswhose components depend on this component.

The component team approach works very well in small- to medium-sized software systems,but is the cause of several of the problems discussed earlier in this section. In particular, usingcomponent teams as the organizing principle may, among others, cause the following challenges:

• Coordination overhead: A customer-facing feature is often broken down into new requirementson several components. This requires the teams to coordinate the precise implementationand interface changes as well as the timing and sequencing of the implementation of theserequirements in order to maintain system stability. In addition, the teams jointly work tobuild and integrate the overall system periodically and need to coordinate their work, oftenrelated to bug fixing in response to interface mismatches and incorrect assumptions about theimplementation by other teams. Consequently, teams need to connect both with the centraldevelopment organization and with other teams during the requirements stage, the developmentstage as well as the integration stage and the amount of overhead associated with this easilybecomes staggering.

• Disconnect with customer: As teams are focused on their component, few team members havea good understanding of the customer’s use of the overall product and their role in deliveringan optimal experience for the customer. This inward focus easily leads to disconnectionand points of stress in the organization between the customer facing product managers anddesigners and the engineers responsible for maintaining the quality of their component.

• Slow response times: The breakdown of a customer-facing feature into requirements formultiple components in the system combined with the lack of a holistic understanding bymany of the engineers, results in a slow response time for software development. The featureis not finalized and ready for shipment to the customer until all teams have developedtheir functionality and the integration of that functionality has been completed successfully.Mismatches in sequencing, misunderstandings between teams, etc. easily cause delays andunpredictability.

The above challenges cause organizations to put strict, top-down processes in place in orderto cope with the lack of predictability. Unfortunately, this will only increase the overhead in thesystem.

As we discuss later in the paper, there is an alternative organization principle, i.e. the use offeature teams that address these concerns. Although it does introduce its own set of challenges,especially in the case of large-scale development, this approach can have fewer evils associatedwith it.

3. RESEARCH METHODOLOGY

We performed a single case study [17] in which we held 14 semi-structured interviews withparticipants of Intuit, a Fortune 1000 company, who were part of the development process. Theinterviews were held when the development process was in place and had been tested for one year(starting 2009). The interviews focused on the old and the new process and the change towards thenew process. These interviewees were on levels from senior management, project management tosenior software engineers. All interviews were taped and transcribed and we collected secondarydata in terms of documentation, presentations, and publications around the development process.



One of the authors was a participant of the organization (complete participation method of Adlerand Adler [18]) and could therefore gain insight into the respondent’s world. Understanding thesetting is of critical importance in establishing research relationships, trust, rapport and in evaluatingboth the information obtained and the interaction that occurs with in-depth interviews [19]. Inorder to increase validation of the interview data, the data were coded by both authors separatelyand compared. Furthermore, the data analysis and results were presented to the case company(respondent validation).

The interviews and secondary data were coded, labeled, and categorized iteratively throughsystematic stages [20] and we found several themes. These themes are discussed in the remainderof the paper.

4. INTUIT QUICKBOOKS PRODUCT LINE

Intuit is a Fortune 1000 software product and services company that serves consumers, smallbusinesses, accountants, financial institutions, and healthcare providers with primarily finance-and accounting-related solutions. Intuit is organized into 10 business units that address differentcustomer segments or provide solutions in specific domains.

The small business group consists of four business units. The largest business unit, FinancialManagement Solutions, owns a flagship product line, Quickbooks, that serves small businesses withan accounting solution. The other three business units focus on payroll, payments, and customermanagement, and to a large extent operate in the Quickbooks ecosystem.

The product line Quickbooks is a market leader in its domain in the United States with a marketshare of more than 75%. The product line supports hundreds of use cases and offers a very richset of functionality that serves small businesses in a host of industries ranging from no-employeebusinesses to businesses with tens of millions of revenue and dozens of employees.

Although Quickbooks is often presented as a product, it is an SPL with several products: SimpleStart, Pro, Premier, Quickbooks Canada, Quickbooks UK, Payroll, Point of Sale, Mac and Online.Further variability is provided by support for specific types of businesses and industries. Theproduct line is largely organized as a configurable product base [7], meaning that the differentproducts can be automatically derived. However, for some of the products, the platform code ismanually integrated with product-specific functionality. The Quickbooks product line also formsthe basis for a rich software ecosystem consisting of thousands of independent solutions providers(ISVs) as well as application developers that build and sell functionality extending Quickbooksfor specific domains.

Up to now, Quickbooks have employed a yearly release schedule where next year’s versionis released in the September timeframe. Development was geared towards this release date andcould be roughly divided into three periods, i.e. feature selection, development, and integration,including quality assurance.

The size of the code base for the main set of products in the product line is between 5 and 10MLOC, and the oldest part of the code is 20 years old. The size of the development organizationhas fluctuated significantly over the years, but is currently around 100 people divided over multiplelocations in the U.S. and India.

Although significant effort has been put into architecture refactoring, the code base has asignificant level of design erosion that complicates the introduction of significant new functionality.

4.1. Product-line challenges

Although not all the symptoms described above in Section 2.1 (coordination overhead, slow releasecycle, and high system level density) were present in this case study, there were several productline-related problems that needed to be addressed. The interview data categorized the problemsand issues with the original approach into four main areas:

• Lack of customer feedback: The original development approach assumed a yearly releasecycle in which during the fall the features to be added were selected, development took place



in the January–June timeframe, integration and testing started in June and ran through Augustand the products were released in September. As a consequence, the product managers anddevelopers involved in a selected feature would only get customer feedback on the featureand its implementation after the release of the product. Furthermore, customer feedback wasdifficult to incorporate as the development organization had already committed to a new setof features for next year’s release by the time the feedback became available. 23% of theinterviewees raised the lack of customer involvement, influence, and feedback as the keydisadvantages of the original development approach.

• Heavy, top-down development process: Owing to the complications of the integration phase interms of required effort as well as its unpredictability, over the years more and more processesand process steps were added, resulting in a heavy, top-down process that left engineersfeeling disempowered and which significantly decreased the efficiency of development dueto all the checks and balances. Of the interviewees, 52% raised the development process asthe key inhibitor for more efficient and effective development and innovation.

• Inefficient use of engineering resources: Owing to the process, development effectivelytook place only during 50% of the year. During the fall, the engineers were recoveringfrom a very intense integration phase, did some minor refactoring, fixed bugs, and releasedpatches. During the integration stage, engineers were primarily concerned with bug fixingand adjusting their components to other dependent components. Although all engineerswere very busy, the efficiency of work was very low. Of the interviewees, 18% consid-ered the inefficient use of resources to be the key challenge with the original developmentapproach.

• Low engagement of team members: The lack of contact with customers, the constrainingprocess, and the inefficiency of resource use negatively affected engagement and motivationof the engineering population. The engineers felt constrained and not in control of their workas they lacked autonomy. As a consequence, the initiative by engineers decreased, resultingin lower productivity and quality. Of the interviewees, 7% raised engagement as the keychallenge of the original development approach.

Although it was clear that the original development process had reached the end of itsuseful life, there were some positive aspects of it that were mentioned by some of the inter-viewees. First, the process had been in place for many years and consequently the wholeorganization knew it inside out. Second, some engineers appreciated knowing all functionalitythat had to be developed at the beginning of the development effort. Finally, as the devel-opment organization was organized around component teams, there was significant expertisethat had been accumulated in key individuals who acted as a resource for the componentteams.

5. NOVEL APPROACH TO PRODUCT LINE ENGINEERING

The goal and contribution of this paper is to present a novel approach to large-scale SPL engineering.This approach combines elements from agile, design thinking and self-selecting, -directing, and-managing teams to achieve customer-focused development by small, independent teams. In thisapproach teams are facilitated teams that (semi-) automatically insert the fruits of their work intothe code baseline after certain milestones have been met.

The approach consists of two iterative processes, i.e. the concept development process wherefunctionality identified as important by customers is developed in an interactive fashion and theproduct release process where all contributions by the various teams are combined, tested, andreleased to customers. Figure 1 gives an overview of the new process.

The remainder of this section is organized as follows. First, we introduce the concept developmentprocess of the team. Then we describe the product release process for the organization. Finally,we summarize the approach and discuss the key benefits of the novel approach for the caseorganization.



Figure 1. Case study process overview.

5.1. Concept development process

The concept development process of the software team is focused on developing new functionalitythat is prioritized by customers and implemented in a way that customers care about. The conceptdevelopment process consists of four main phases, i.e. the solution jam, the code jam, developmentiterations and, finally, solution validation.

5.1.1. The solution jam. The solution jam is organized at the product development organizationlevel. It is a full-day event where 10–15 customers are invited and all PM and PD staff currentlynot full-time assigned to development projects attend. Also, senior PM and PD management ispresent. In total, somewhere between 20 and 50 Intuit staff attend the event.

Before the solution jam, the coordinator connects with individuals and already created teams tofacilitate the formulation of ‘customer pain’ statements, i.e. hypotheses about areas that customerswould prioritize development to take place in. These customer pain statements are quite concreteand specific and are used as the starting point for conversations and reviews with the customerspresent at the solution jam.

The customers invited to the solution jam are selected based on their potential interest in theareas addressed by the customer pain statements. For instance, if a customer’s pain statementaddresses support for multi-location inventory management customers are selected whose businesshas multiple locations.

Teams can be formed before as well as during the solution jam. Feature teams are formed that seetheir own copy of the code base line allowing them to make changes necessary for implementingtheir feature, anywhere in the system. The team formation is completely bottom-up and is notinfluenced by management. The average size of the team is three members, but it can fluctuatefrom a single individual to seven people. It is important to note that the teams are self-selected, i.e.management does not influence their formation, self-directed, i.e. the team selects the customerpain that it wants to work and are self-managed, i.e. there is no standard division of tasks. Thelatter is interesting as team leadership can land with individuals in different roles and may evenchange over the course of the project as different skills become predominant.



During the solution jam, the team fleshes out the customer pain statement and starts to designa solution. This solution is not implemented in code, but is expressed in terms of ‘mock-ups’ andsketches and complemented with verbal commentary. Once a first rough draft design has beencreated, it is presented to a few of the attending customers for feedback. Based on the feedback,the team continues to iterate the design.

The customer feedback at this early stage in the process is invaluable to refocus teams and tomake sure that teams are prioritizing the right aspects of the solution. Although Intuit engineerscollectively spend more than 10 000 h per year in ‘Follow Me Home’ activities where customersare followed and observed in their daily routine (a form of ethnography), product developersstill do not have the same level of understanding of the needs and wishes of customer than thecustomers themselves. Receiving customer feedback at the very start weeds out the well-intendedbut ill-conceived hypotheses of customer ‘pains’ before any significant investment is made.

The second reason is that even if the customer pain is correctly identified and significant,engineers may still focus on the wrong aspects initially rather than fleshing out the design of themost important aspects. Immediate feedback and input from customers will cause the engineersto focus on the right aspects of a solution immediately.

During the day, the team iterates on their concept, receiving input and feedback from customers.During the final hour, all teams get the opportunity to present for a panel consisting of engineeringand product management as well as customers. The panel decides which of the presented conceptsare considered to be the most viable. Typically, the panel accepts around 50% of the concepts toproceed to the next stage.

5.1.2. Code jam. The next stage in the development process is the code jam. Often organized in thesame week, with perhaps a day in between, a two-day code jam is organized where the teams withaccepted concepts from the solution jam get the opportunity to build a skeleton implementationof the solution.

As discussed earlier, the Quickbooks code base is around two decades old and hence significantdesign erosion has occurred over the years. This has caused the creation of a number of beliefs in thedevelopment organization about what things are hard to change and which are easier. Rather thanrelying on the beliefs in the organization, the code jams validate the technical complexity throughthe implementation of a skeleton solution. The skeleton provides a partial implementation of themost basic functionality while focusing on assessing the highest risk areas of the implementation.

The skeleton implementation provides deep insight into the complexities associated with a fullimplementation of the concept and highlights the key areas where difficulties can be expected. Theskeleton implementations are also used for taking a first peek at what the solution will look likefor the customer. Even if a concept is visualized during the solution jam using mock-ups, seeingthe implementation in software is still significantly more informative.

The code jam ends with a panel presentation from each of the participating teams. The panelconsists of product and engineering management, and assesses the estimated value of each of theconcepts as well as the predicted cost and complexity of implementation and integration. Basedon this information, the panel selects the highest ROI (return on investment) concepts. In a typicalcode jam, the panel selects around half of the presented concepts.

5.1.3. First development iteration. The teams whose concepts were accepted at the code jamreceive the go-ahead for a first two–three week iteration. The team now has much more time thanduring a solution or code jam to start developing a proper implementation of the concept in thecontext of the Quickbooks product line. Depending on the target audience of the concept, it iseither implemented directly in the platform or in a specific Quickbooks product.

The purpose of the design iteration is to prove a specific hypothesis about the value and usabilityof the concept. As this obviously requires the feedback from customers, the team is expected tohave collected customer feedback before the end of the iteration. The customer feedback is oftencollected through telephone meetings with customers where a screen-sharing tool is used to showcustomers the implementation of the concept in order to receive feedback.



At the end of the first iteration, the current implementation of the concept as well as the customerfeedback is presented to product and engineering management. The decision at this point is oneof three alternatives. First, the team can get the go-ahead for a subsequent iteration. Second, theteam can be stopped and dispersed if the results of the implementation and the customer feedbackare not satisfactory. Finally, although unlikely at the end of the first iteration, the team may havefinalized the implementation to the extent that the concept can be integrated in the Quickbookscode base. The acceptance rate for concepts is much higher once the team has been allowed toenter its first iteration, hence in practice most teams proceed with implementation of their conceptuntil its completion.

5.1.4. Subsequent development iterations. Once the team can proceed after the first iteration, theprocess is similar and repeats the described process. For every iteration, development takes placeto test a specific hypothesis related to the concept with customers, customers provide feedbackand at the end of the iteration, the results are presented to product and engineering management.

For larger and more complicated concepts, though, there is an additional mechanism forcollecting customer feedback: the feature alpha. Although customers can provide a certain level offeedback through screen sharing and telephone conferences, the concept cannot be used in theirday-to-day practice. For this purpose, the notion of a feature alpha is introduced. The feature alphaconsists of the most recent stable and released version of Quickbooks extended with the conceptimplementation. As the feature alpha contains mature Quickbooks code, the customer can use thefeature alpha in their day-to-day operations. The customer can also use the functionality associatedwith the concept, but with the understanding that the code is not yet of production quality andhence may experience stability issues.

The feature alpha allows the customer to provide feedback on the concept in the context of theirreal business. Also, customers are selected because of their specific interest in the functionality.For instance, to evaluate multi-site inventory functionality, customers are selected that already havemultiple sites today where inventory is held and hence are keen to see that aspect of their businessbetter supported by Intuit’s solution.

Once several iterations have been invested in the concept, the concept will most likely beincluded in the product line. Thus, the typical exit point for the concept and the associated team isthe end of an iteration where the concept has received positive customer feedback and is consideredcomplete by product management. At this point, the concept is integrated into the product lineand the team disbands and returns to the pool of engineering resources. Of course, the engineersimmediately start to look for new ‘customer pains’ to bring to the next solution jam and to formteams around ideas based on their passions and interest.

5.2. Product release process

The previous section described the bottom-up part of the product development process. In thebottom-up process, the teams identify, present, and develop new concepts. There is also a top-downpart of the process where product management identifies a small number of typically larger featuresthat need to be developed in addition to the concepts identified as part of the bottom-up process.The engineering resources for the top-down part are selected and assigned by the management,rather than following the principles of self-organization.

The top-down process, as described earlier, is how Quickbooks was developed earlier andone could assume that this is relic from earlier days. However, there is broad agreement in theorganization that there is a need for both types of development in the context of Quickbooks inthe future as well. The top-down development process has, however, been extended with customerfeedback in different stages of the development.

As part of the overall push to collect more and earlier customer feedback, the developmentteam creates several product alphas during the year. A product alpha is similar to a feature alpha,except that it combines several features and concepts into one release that is distributed to selectedcustomers. As a product alpha contains so many new features, it is significantly less stable thanfeature alpha and hence customers in their daily operations cannot use the release. Instead, the



product alpha is released embedded in a virtual machine (to avoid installation issues) and operateson a copy of the customer’s Quickbooks data file. In this fashion, the customer can test theproduct alpha in the context of his or her own business without risking corruption of the businessdata.

At this point, the Quickbooks desktop versions are released once per year. As a consequence,towards the end of the development cycle more and more resources are transitioned from devel-opment to testing, validation, and bug fixing. As part of the future evolution of Quickbooks, theintention is to release new functionality quarterly or monthly. The capabilities for achieving thishigh a frequency of releasing production quality code are being developed as part of the transitionto the new development model.

5.3. Benefits of the new approach

From the interviews and secondary material from the case study we found a number of benefitsfor the new development approach. These benefits are discussed below.

• Customer feedback: Throughout the new approach, the customer is deeply involved in allaspects of the development, starting from the earliest ideation stages throughout the matura-tion of new functionality. The solution jam, customer feedback during development iterations,the feature alpha and the product alphas all contribute to continuous customer feedback.In the earlier stages, the customer actively helps shape the development of new function-ality, whereas in later stages the feedback is more focused on qualitative and quantitativefeedback on what has been built. From the interviews 22% of the interviewees mentionedthe importance of customer feedback. Especially the customer engagement and feedbackwas highly valuated. Other aspects mentioned during the interviews were the fact thatcustomers were able to test or experiment features early on and many mentioned the positiveresponses from customers on the developed solutions as validation and motivation for theirwork.

• Development process overhead: The new process is heavily focused on minimizing the depen-dencies between different teams as well as between the central development organization andthe development teams. This has resulted in a situation where teams have a great deal ofautonomy between the milestones in the process, i.e. at the end of the solution jams and thecode jams, at the end of each development iteration and throughout the integration of theircode into the software baseline. A major supporting factor in this case is the transition fromcomponent teams to feature teams. As feature teams effectively have their own developmentcode line in the SCM system, each team can make the necessary changes in all the compo-nents without having to coordinate with other teams until the integration stage. The featureteam solution was discussed during the interviews as empowering and motivating for teammembers.

• Use of engineering resources: Development of new functionality now takes place throughoutthe year, rather than just during the winter and early spring of the year. This means thatdevelopment, QA and integration resources now to a large extent work in parallel rather thanin a sequential fashion. In addition to significantly increasing the effective use of engineeringresources, it also prepares the organization for its transition to a continuous release model.In the interviews many interviewees mentioned the increase in speed (9% of interviewees) indevelopment.

• Engagement: The amount of autonomy, the avoidance of a yearly peak during the tradi-tional integration stage and the frequent customer interactions have significantly increased theengagement by engineers. Team members and managers mentioned during the interviewsthe increase in engagement and motivation of team members. Other aspects mentioned duringthe interviews were: team members can have ownership of features; work in cross-disciplinary,small and organic teams that can learn from each other.

In the following section, we evaluate the approach with respect to the key challenges identifiedfor the original approach for SPL engineering in general.



6. DISCUSSION AND CONCLUSION

In the sections above, we have introduced a novel approach to building large-scale software inthe context of SPLs which uses elements from design thinking, agile software development, andself-organizing teams. The approach was developed in response to a number of key challengesthat SPLs start to suffer from over time, i.e. high coordination overhead, slow release cycles, andincreasing error density.

In the case of the case study company, the generic problems in legacy SPLs translated themselvesinto four main challenges, i.e. the lack of customer feedback during development, a heavy, top-down process that had evolved in response to lack of predictability, this process had in turn resultedin inefficient use of engineering staff which negatively affected the engagement levels of engineersnegatively. The new approach resulted in four important key elements:

1. Self-selected, directed, and managed teams: A major part of R&D is organized into smallteams that self-select their members, self-direct (based on customer input) their work and toa large extent, manage themselves.

2. Continuous customer involvement: Customer involvement in this approach is raised to a newlevel. In most products, customers were especially used to provide feedback after devel-opment was complete. In this case, customers are involved throughout the developmentprocess.

3. Solution and code jams: One and two-day jams are an integral part of the developmentprocess and have a dedicated purpose, i.e. converting an idea into a real, partially validated,concept and exploring the complexities of implementing it in the legacy code base.

4. Feature alphas: The ability to present the customer with a stable version of the most recentrelease of a product in the SPL but extended with a specific feature that is under developmenthas proven to be a particularly powerful customer feedback mechanism.

The topic and contribution of this paper is twofold. First, we presented the main issues of SPLand agile software development methods for large-scale SPL engineering. Second, we presented anovel approach to SPL engineering that encompasses agile, design thinking, and self-organizationprinciples and illustrate the approach by means of a company case study. The case study is anillustration of a compositional SPL. Compositional SPLs replace centralized decision and coordi-nation processes with significantly increased autonomy of teams, the use of the SPL architectureas the coordination mechanism and a much stronger focus on compositionality of componentsand backward compatibility of interfaces. These practices allow significantly reduced productderivation effort and greatly simplified post-deployment evolution of products [3, 4]. Importantprinciples in compositional SPL are the fact that teams internalize the company strategic roadmapand can decide on the content of their next release. Furthermore, new component releases arebackward compatible with the previous release. Teams never build on functionality that is stillunder development, but release frequently and predictably.

Although the novel SPL engineering approach presented in this paper has addressed the keyissues that it intended to solve, there are still a number of areas for future discussion and research.This product line approach helps to deal with the development of large-scale software throughagile and design thinking principles, however, these process changes also have an important impacton the software development organization, the way its management is set up as well as a variety ofother topics, including human resourcing issues such as performance management. Next to processchanges, managerial roles also changed in this example, i.e. the role of the manager changed withinthe new process to a coaching role in which managers set the right expectations for compensation,failure, and team dynamics.

One topic for future research is the long-term implications of a transition to self-selected,self-directed, and self-managed teams. How team members are selected, having the right teammembers in your team, dealing with available people resources, and management expectationsneed to be studied more in the future. Furthermore, employee performance as well as managing theindividual’s perception of adding significant value especially when being part of several solutionjams without being selected for subsequent iterations is an important topic of study.



Another important topic for future research is the management of architecture evolution andrefactoring without sacrificing the principles of agility and self-managed teams. Agile methodsoften tend to de-prioritize architecture, leaving architecture erosion as a consequence. However,for compositional SPL engineering to be effective, the architecture plays a central role.

The customer engagement and involvement early on in the development process has been verybeneficial for this particular case study. However, an important aspect for future research is howsoftware companies receive input from the right customers. Selecting the right customer for aparticular concept or feature at the right time in order to optimally benefit from the customer inputand validation is a complicated problem. Finally, in the context of large-scale SPL developmentfor mass-markets, translating the input from a small selection of customers into accurate insightsthat apply to the entire customer base is a potentially complicated problem of induction.

ACKNOWLEDGEMENTS

The authors thank the interviewees at the Intuit Quickbooks development organization, in particular AtticusTysen.

REFERENCES

1. Bosch J. Design and Use of Software Architectures: Adopting and Evolving a Product Line Approach. Addison-Wesley: London, 2000.

2. Clements P, Northrop L. Software Product Lines: Practices and Patterns. Addison-Wesley: Boston, 2001.3. Bosch J. Toward compositional software product lines. IEEE Software 2010; 27(3):29–34. DOI:

ieeecomputersociety.org/10.1109/MS.2010.32.4. Bosch J, Bosch-Sijtsema PM. From integration to composition: On the impact of software product lines, global

development and ecosystems. Journal of Systems and Software 2009; 83:67–76. DOI: 10.1016/j.jss.2009.06.051.5. Boehm B, Turner R. Balancing Agility and Discipline—A Guide for the Perplexed. Addison-Wesley: London,

2004.6. Bosch J, Bosch-Sijtsema PM. Coordination between global agile teams: From process to architecture. Agility

Across Time and Space. Implementing Agile Methods in Global Software Projects, Smite D, Brede Moe N,Agerfalk P (eds.). Springer: Berlin, 2010; 217–233.

7. van der Linden F, Bosch J, Kamsties E, Kansala K, Obbink H. Software product family evaluation. Proceedingsof the Third Conference Software Product Line Conference, SPLC’04 (Lecture Notes in Computer Science, vol.3154). Springer: Berlin, 2004; 110–129.

8. Beck K. Extreme Programming Explained: Embrace Change. Addison-Wesley: Boston, 1999.9. Poppendieck M, Poppendieck T. Lean Software Development: An Agile Toolkit. Addison: Boston, 2003.

10. Rising L, Janoff NS. The Scrum software development process for small teams. IEEE Software 2000; 17(4):26–32.DOI: 10.1109/52.854065.

11. Schwaber K. Agile Software Development with Scrum. Prentice-Hall: New York, 2001.12. Larman C. Agile and Iterative Development: A Manager’s Guide. Addison-Wesley: Boston, 2004.13. Babar MA, Ihme T, Pikkarainen M. An industrial case of exploiting product line architectures in agile software

development. Proceedings of the 13th International Software Product Line Conference (SPLC’09), San Francisco,CA, vol. 446, 2009; 171–179.

14. Hanssen GK, Fıgri TE. Process fusion: An industrial case study on agile software product line engineering.Journal of Systems and Software 2008; 81(6):843–854. DOI: 10.1016/j.jss.2007.10.025.

15. Kruchten P. Software architecture and agile software development: A clash of two cultures?Proceedings of the 32nd ACM/IEEE Conference on Software Engineering 2010; 2:497–498. DOI:http://doi.acm.org/10.1145/1810295.1810448.

16. Tian K, Cooper K. Agile and software product line methods: Are they so different? First International Workshopon Agile Product Line Engineering (APLE’06). IEEE Computer Society: Baltimore, MD, U.S.A., 2006.

17. Eisenhardt KM. Building theories from case study research. Academy of Management Review 1989; 14(4):532–550.

18. Adler PA, Adler P. Observational techniques. Handbook of Qualitative Research, Denzin NK, Lincoln YS (eds.).Sage: Thousand Oaks, 1994; 377–393.

19. Miller J, Glassner B. Interviews. The inside and the outside, finding realities in interviews. Qualitative Research.Theory, Method and Practice, Silverman D (ed.). Sage: London, 1997; 99–112.

20. Corbin J, Strauss A. Basics of Qualitative Research (3rd edn). Sage: Thousand Oaks, 2008.


Documents

Introducing agile customer-centered development in a legacy software product line