world quality - Sogeti UK | Software Testing Services ... · The 2010-2011 World Quality Report is a result of ongoing collaboration between Capgemini Group and HP Software & Solutions

worldqualityreport

2010-11

Preface 2

Foreword�by�the�Authors 3

Executive�Summary 4

Introduction 5

Testing�Tools�&�Technology 6

Testing�Organization�&�Tester�Profile 11

Outsourcing�Trends 12

From�Agile�Development�to�Agile�Delivery 14

Deploying�Applications�on�the�Cloud 18

Industry�Trends 20

Economic�Impact��32

Summary��34

About�the�Study��35

About�Us��36

contents

2

Raf HoweryVP, Global Channels & Partners Executive, Capgemini

We are pleased to bring you the 2010-2011 edition of theWorld Quality Report. This report is designed to helpcompanies understand the trends in application quality andtesting across different geographies as well as industrysectors. The report also presents best practices to helpquality assurance organizations learn how to do things better,quicker, and cheaper.

We believe that it is important to continually developmarket-driven solutions that meet our clients’ qualityassurance demands. In this report, we monitor business,technology and economic trends in the software quality andtesting space. This year, we added data on the direction of ITinvestments and application quality in a number of specificindustries: Consumer Products and Retail, Distribution andLogistics, Financial Services, Public Sector, High Tech andTelecommunications. We hope you find this report to bevaluable and that you gain insights into an increasinglyimportant aspect of the Application Lifecycle Managementprocess. In our next edition, we intend to increase thestatistical sample and create an associated tool that allowscompanies to benchmark themselves against their peers inthe same market segment and other segments who may bemore mature, thus improving the learning process.

We want to thank HP for its continued collaboration and forhelping to make this study possible. Capgemini Group andHP share a common culture, so it is natural to combine thedepth and breadth of our business intellectual property withthe strength of HP in the testing market. Introducing theWorld Quality Report is just one of many collaborationpoints between Capgemini Group and HP in our ongoingefforts to provide guidance to our clients.

Jonathan RendeGeneral Manager & VP, Application Business, HP Software & Solutions

HP’s Application Lifecycle Management business has longbeen a thought and market leader in the QualityManagement sector. Our continued investment in qualitymanagement, requirements management, service testing andagile process support is based upon a deep understanding ofindustry shifts and trends. We are very proud to team withCapgemini and contribute to the new World Quality Report.

We believe that the role of quality is increasing in status andbusiness relevance. It is no longer just a gate in the softwaredevelopment process. It is now, arguably, the top ingredientfor business success for an IT organization, that begins farupstream in demand and requirements management andextends far downstream into production. There is no doubtthat quality is a top CIO priority. Quality organizations nowget involved earlier, have greater decision making ability andhave far more business responsibility when representing theend user community.

Profound changes are driving how applications are built anddelivered. Mega trends such as process and organizationalshifts driven by the agile movement and cloud computingare increasing in visibility and implementation. While theworldwide business climate is showing some signs ofimprovement, I see that more organizations are leveragingthe uncertain economy to transform their IT applicationportfolios in the best way possible.

I want to again thank Capgemini for the invitation toparticipate in the 2nd edition of this report. We are proud toco-sponsor this project and see how the above changes willcontinue to drive positive change in the application industry.Most importantly, I’m excited about how organizations cannow find better ways to deliver their applications.

Preface�•

3

2010 has brought many changes. The economies around theworld are slowly on their way to recovery, IT spending is onthe rise, and companies are starting to reinvest in modernizingtheir applications while software quality is once again movinginto the spotlight.

What the recessionary years have taught us is that often theonly way to thrive in a difficult economic climate is to findways to cut operating costs, while increasing speed andefficiency. To do that, companies need IT systems andorganizations that can adapt quickly to changing marketconditions, perform without interruption and providesecurity and data integrity that’s expected in today’s business world.

With a combined 40 years of experience in QualityAssurance (QA), Capgemini and Sogeti are often asked tohelp companies develop and implement quality processesthat would help ensure their IT applications’ reliability andstability. Since 2008, Capgemini Group and HP – recognizedleaders in application quality – began conducting a formalglobal survey and publishing a yearly World Quality Reportto help clients stay on top of the latest trends in applicationquality, methodologies, tools and processes.

In this year’s report, we revisit quality trends and approachesand take a closer look at related industry-specific issues andchallenges. We also examine new trends in applicationdelivery – such as using agile methodologies and testingcloud-based applications – and explore the effects that thesenew approaches are having on QA practices and techniques.

We are committed to continuing this tradition for years tocome – giving our readers a closer look at software qualitytools, the state of quality, changes in QA organizations,outsourcing and the recent economic effects in the testingspace. We would like to thank all IT professionals who tookthe time to complete the survey and contribute to thisreport. We would also like to recognize contributors from HP and Capgemini Group for their immense support anddedication to this research.

We hope you find this report engaging and beneficial to yourcompany’s continued success.

Welcome to the 2010-2011 edition of the World Quality Report

Murat Aksu Global Head of HP Software Alliance, Capgemini

Charlie Li VP, Global Testing Services, North America Leader, Capgemini

Foreword�by�the�Authors�•

world quality report 2010-11

4

The 2010-2011 World Quality Report is a result of ongoingcollaboration between Capgemini Group and HP Software &Solutions. It presents findings from the 2010 global survey ofCXOs, IT directors, quality assurance managers and engineers,and examines the state of application quality and testingpractices across different industries and geographies.

Each year, this report looks at current trends in softwarequality and examines emerging tendencies that may affect the future of testing. Survey respondents are asked toevaluate the level of investment in Application LifecycleManagement (ALM) solutions in their organizations, rate thereturn on investment for these solutions, answer questionsabout their testing practices, and assess the general levels ofapplication complexity. This year, the survey also includedspecific questions about the adoption of agile delivery andtesting methods, and application virtualization.

Key FindingsThe research revealed a number of key findings:

•Although in 2009 most companies had to reduce their ITspending, the vast majority of organizations (71%) are stillinvesting in new application development projects. ITinvestments are shifting from a daily, operational focustowards building new applications that bring competitiveadvantage for the business.

•Developing new IT systems with limited resources meansthat both developers and testers experience an increasedworkload. Half of the survey respondents indicate thatresource cuts and heavier project loads have forced theirproject teams to take on more work. Increased pressuremandates the need for greater efficiency, more consistentQA methodology and better reuse of automated testing assets.

•Having a defined application lifecycle methodology is nolonger viewed as a barrier for agility. More than 70% of allsurveyed organizations admit to following a specific ALMmethodology, and 82% of respondents indicate that testingprocesses are being consistently followed in half or more oftheir IT projects proving that agile methodology and testingprocesses are not mutually exclusive.

•QA organizations increasingly require well-roundedresource skill sets. While resources with strong testing skillsand business knowledge are always important, anincreasing number of teams prefer to bring on board testers

who also have a good understanding of developmentpractices and methodologies. Close to 72% of all surveyedIT professionals say that testers with developmentbackgrounds perform their tasks better than those withoutthe technical expertise. This shows a trend for greaterconvergence between the business, development and QAskill sets, and an increasing need for testers to take on amore active role in both the business requirement anddevelopment processes.

•This is especially true for agile organizations. Agile isrequiring the teams to work with a greater degree ofcollaboration. QA can ensure that quality becomes part ofthe application lifecycle by providing the right skill set –technical knowledge of the development process and strongbusiness process expertise.

•Agile delivery methods continue gaining popularity. Over60% of surveyed companies leverage agile, although mostare still experimenting, utilizing agile only in a portion ofnew IT projects. This approach allows organizations toadopt agile gradually. With this approach, they can identifydevelopment and testing practices that work well for them– rather than switching a large number of projects to agilewithout considering the implications on their organizationalstructure, available skill sets and existing practices.

•Organizations that have adopted agile see a significantimprovement in time-to-market and application quality,although there are still challenges – specifically infacilitating communication and collaboration between teams.

•Communication is also noted as the number one barrierpreventing agile organizations from outsourcing applicationdelivery. However, the trend for outsourcing is not affectedby an organization’s development methods. Nearly 50% oforganizations outsource application development andtesting whether they use agile or utilize more traditionalmethods. Additionally, over half of organizations areplanning to increase the use of offshore resources in thecoming year. The biggest growth is seen in Europe, wherequality management outsourcing is outpacing other regions.

•Along with improved agility in software delivery, ITorganizations are aiming to gain agility in their applicationsand infrastructure. Virtualization enables companies toimprove capacity planning and save significant resources onhardware and datacenter maintenance. Nearly three quartersof all respondents are starting to leverage deployment of some

Executive�Summary�•


5

of their applications on the cloud, with only 8% experiencingproblems that caused them to roll back to physical infrastructure.

•Cost reduction is the number one reason for companies tomove their applications to the cloud, followed by increasedagility. However, many companies are still unclear how toleverage cloud services for application testing. Cloudinfrastructure could offer many advantages in quicklybuilding up test environments, hosting testing tools on thecloud, or generating load for performance testing. Webelieve that in the coming years, more companies willbegin to realize these benefits, and we will see an evengreater adoption of cloud services.

To keep pace with current application quality trends andtechnologies, Capgemini Group is surveying thousands of ITprofessionals from around the world on an annual basis toassess the current state of application quality, how companiesperceive their application quality and how quality is affectedacross different industries.

This year’s World Quality Report examines the effects of therecent economic downturn on companies’ IT investments,identifies emerging trends in using test automation tools andtesting methodologies, looks at the changing profile of a QAtester and analyzes current trends in QA outsourcing. Thereport also explores new technologies that have a directimpact on quality – agile application delivery and cloudcomputing. The 2010-2011 report also includes a detailedprofile of the state of IT as it relates to quality in six specificindustries:

•Consumer Products and Retail

•Distribution and Logistics

•Financial Services

•Public Sector

•High Tech

•Telecommunications

We hope that the data points provided by the report –together with our experts’ opinions – will help you gain abetter understanding of the current state of applicationquality and the trends that affect it.

Introduction�•

IT organizations areaiming to gain agility intheir applications andinfrastructure.

Application complexity is increasing, but quality is also on the riseToday’s IT applications support increasingly complexbusiness processes. Not surprisingly, applications themselvesare becoming more intricate and inter-connected. In oursurvey, an overwhelming majority (88%) of surveyrespondents agree that software systems have increased incomplexity. In the past, organizations have been able to dealwith increased workloads by adding resources to theirquality management teams, but with reduced IT spending,companies are looking at alternate ways of keeping pace withgrowing complexity – such as streamlining the qualityprocess, increasing the degree of test automation andoutsourcing testing activities.

However, despite poor economic conditions and limited QAresources, IT organizations still seem to see a steady increasein software quality over the last two years. Over 80% ofrespondents say that quality is actually improving. We attribute this to the fact that a growing number oforganizations are treating application quality as a formalprocess – incorporating quality methodology into theirapplication lifecycle. Nearly three quarters of respondentsadhere to a defined QA methodology that they follow in themajority of their IT projects. We see this as a sign thatapplication quality will continue to improve and keep upwith the increased complexity of business applications.

Businesses use more test automation, but struggleto realize full ROI on their solutions The use of test automation technologies is on the rise.Companies are increasing their investments in qualitysolutions and are becoming more skilled at using them. Our survey found that nearly half of all businesses reuse50% or more of their testing assets, ultimately leading toincreased testing efficiency, higher levels of test caseautomation and shorter testing cycles.

However, a more detailed analysis reveals that testautomation technologies are more widely leveraged in thelater stages of the application lifecycle, such as test casemanagement and defect tracking, and not as commonly usedin the earlier stages, such as requirements gathering, usecases, and business flow modeling. (See Figure 1.)

Defect tracking and test management represent moretraditional QA tasks, and automation tools for these phaseshave been available for many years. Solutions for the earlierstages of the ALM are still relatively new on the market andrepresent an innovative approach to application delivery –involving QA in all stages of the lifecycle. A growing numberof companies recognize that the cost of fixing problems latein the development phase is much higher than the cost ofpreventing problems by incorporating quality intorequirements and use cases. Yet, they are still slow to adoptquality solutions for the entire lifecycle – largely due to theirinability to accurately quantify Return on Investment (ROI)on the solutions that they already own.

Q: What components of your organization’s ALM processleverage automation technology?

Figure 1: Companies are slower to adopt automationtechnologies for earlier stages in the application lifecycle.

6

All of the above

Business IP-domain mgmt

Business flow modeling

Requirements

Use cases

Test cases

Defect mgmt

}

7

Companies are increasingtheir investments in qualitysolutions and are becomingmore skilled at using them


Nearly half of all survey respondents are unclear as to theextent to which purchased ALM tools’ licenses are beingleveraged and whether they are generating the expected ROI.(See Figure 2.) Across all regions, only 4% of IT professionalsagree that all of their ALM investments are fully paying off,and only slightly more than a third of respondents said thathalf or more of their solutions have been fully implementedand are helping to improve application quality. QA organizationsneed to focus on increasing the adoption rates of theirpurchased technologies. This will help demonstrate positiveROI and measurably improve their ability to secure fundingfor additional investments in automation technologies,especially for the much-needed solutions in the earlier stagesof the ALM. Establishing formalized measurement programswill enable QA organizations to track relevant metrics drivenby business objectives. This will provide data and facts in abusiness context to demonstrate the value of investments inapplication quality.

Not enough resources are invested in tool adoptionOne of the main reasons companies fail to fully leverage theirtest automation solutions has been the lack of planning andinadequate funding needed to deliver key services aroundthe deployment of automation tools, training and mentoringof resources. (See Figure 3.)

8

100%

Up to 80%

Up to 50%

Up to 25%

Can’t quantify

Not enough resources are invested in the adoption of the technologies

No management support

Lack of internal process

Lack of integration among different technologies

Technologies too difficult to fully adopt and/or lack of expertise

Wrong choice of technology

Q: What percentage of your purchased ALM tools’ licensesare being used and generating the expected ROI?

Figure 2: Only a small number of companies believe that all oftheir ALM tools’ investments are generating expected ROI.Almost half cannot quantify the return on investment.

Q: What are the biggest obstacles that prevent you fromfully leveraging 100% of your ALM tool investments?

Figure 3: Limited investment in adoption and lack of internalprocesses are leading the list of obstacles that preventcompanies from fully leveraging their investments in ALM tools.

}

}

In order to cut costs at the time of technology acquisition,purchasing departments often veto service-line items thatwould help with the planning and successful implementationof enterprise-level technologies. In some cases, theorganization may purchase services for a successful initialimplementation, but underfund ongoing maintenance andmentoring, thereby undercutting the future success of theinvestment. In other cases, turnover in resources and projectchampions can cause automation efforts to slow down orcompletely stall, with QA teams reverting back to creatingmanual test cases using word processing applications. Inextreme circumstances, QA teams diligently create thousandsof automated scripts without a clear strategic plan, only tofind that their efforts go to waste due to drastic changes inthe platforms and the functionality of applications undertest. It is important to note that simply learning how to usethe automation tool is not enough. To successfullyimplement a test automation project, it is essential to have adeep understanding of automation strategies, frameworksand tools. In addition, having an experienced championmentoring the project will increase the organization’s chancesof meeting their quality objectives. In practice, it takes atleast three to five years of direct, hands-on experience todevelop automation skills that resemble a champion.

In order to fully leverage their investment in test automation,enterprises need to look at the big picture when acquiringand deploying tools. A well-defined implementation,integration and adoption strategy with qualified professionalservices and training for internal staff can help push atechnology investment to exceed expected ROI and move theorganization closer towards their commitment to quality. QAteams need to learn to look at IT investments in the sameway business investments are considered and implemented.

Companies believe in boosting investment inautomation technologies Survey results indicate that new and increased investmentneeds to be considered for automation throughout ALM,particularly for requirements visualization, testenvironments, data management and business modeling.Investment in these areas is expected to generate higherquality applications, reduce rework, and decrease the cost oftesting and defect management later in the applicationlifecycle. (See Figure 4.)

1. Requirements visualization and business flow modeling

Requirements gathering and management have traditionallybeen a problematic area for the quality process. Most of therequirements documents that are generated by the businessanalyst community have been fairly complex andvoluminous. The fact that most requirements documents donot accurately correlate to the actual, testable businessprocesses limits the ability of QA teams to extensively utilizethem. If QA is unable to understand the requirements, theycannot effectively translate them into corresponding testscripts. The complexity of interpreting and tracingrequirements to test scripts has undermined theestablishment of repeatable testing processes. This hasresulted in ad-hoc testing rather than the requirements-basedvalidation that is the industry best practice.

9

Requirements management

Test management and execution

Environments and data management

Business flow modeling

Defect management

None, we have all that we need

Q: Where would you like to see additional technologicalinvestments made in ALM automation?

Figure 4: Companies see requirements management, testmanagement and environments and data management aspriority areas for investment. }


10

There are new technology vendors, such as BlueprintSystems, that are attempting to bridge the gap between thebusiness analyst and QA teams by transforming requirementsfrom plain text into logical business flows that are supportedby visualization, prototyping and simulation. In some cases,the requirements visualization has been tightly linked to QAprocesses and can automatically generate use cases and testscripts to reduce upfront testing cost and time. Time andeffort should be spent by the business in evaluating wherevisualizations will add the most value.

2. Environments and data management Test environments and data management are becoming agrowing concern for QA teams. As application complexitygrows, it is becoming increasingly difficult for QA teams tocreate testing environments that mimic the real-lifeinfrastructure and environment of production applications.In some cases, government regulations and privacy concernsrestrict the use of production data in testing environments.Technologies such as test environment virtualization anddata masking are increasingly becoming a necessity for QAteams to manage these situations.

IT organizations are more consistently usingcommon methodology, but their testing practicesare still mostly proprietary and not based onindustry standardsThe growth in automation technologies is at least partiallyfueled by the increase in adoption of consistent testingmethodologies. As organizations mature, they realize that ad-hoc testing does not provide the advantage of reusing testingassets, standardizing on common metrics or having arepeatable quality process. This year’s survey indicates that asmany as 72% of IT organizations follow a defined ALMmethodology both in their development process and in theirtesting activities. The number varies slightly between regions,with the lowest number of ALM followers in Europe (65%)and the highest in Asia (78%).

However, when asked to choose from a list of industry-standard methodologies, nearly three quarters of respondentssay that they do not follow Test Management Approach(TMap®), Test Maturity Model Integrated (TMMI) or othercommon standards. Instead, their organizations develop anddocument their own best practices that are followed in themajority of development and testing projects. While having acommon methodology is a positive trend, it is often open tointerpretation. Different groups in the organization mayadopt their own “versions” of the standard practice, and as aresult, the company as a whole is not fully realizing thebenefits of standardization, economies of scale, commonmetrics, unified reporting and asset reusability. This isespecially true for large enterprises that give their divisionsand subsidiaries relative autonomy in choosing their testingtools and practices. Also, many of these homegrown methodsare usually development-focused and do not specificallyaddress testing. Implementing a standard testingmethodology such as TMap® or TMMI requires a company-wide commitment. Only when every IT group within thecompany adopts the standard practices can the companyachieve the quality improvement benefits offered by themethodology. Having a homegrown policy is stillsignificantly better than not following any formal QAmethods, but it may not afford companies the full benefitsand efficiencies offered by a more standard, widely acceptedmethodology.

11

Business is looking for a new tester profileHistorically, testers have been former developers who haveswitched careers to QA or business subject-matter expertswearing multiple hats. Today, the profile of a tester ischanging rapidly. Due to the growing integration of IT andcompany-specific business goals, companies prefer testerswho have both strong technical skills and relevant domainand business knowledge. In addition to being knowledgeableabout automation technologies, testers need to be able tounderstand business requirements and create test validationfor specific business needs. Organizations are also lookingfor testers who can provide actionable analysis to developersfor fast resolution of defects.

More than two thirds of respondents (72%) say that theirtesters have several years of business domain experience andan additional 18% indicate that their testers have full domainexpertise. Business knowledge is the second most valuableasset identified for a QA tester following QA skills. In-depthknowledge of development processes and programminglanguages came fifth in the survey behind database andscripting skills. (See Figure 5.)

This clearly suggests that future testers must become betterrounded in both business domain and technical skills. QA isbecoming a more professional career where organizations areexpecting a smaller, but more skilled and highly developedworkforce steeped in business knowledge. As applicationcomplexity increases, time-to-market challenges will requiretesters to have domain, application and technologyknowledge in order to quickly dissect problems and providefast and accurate information to development teams.

Along with tester profiles, the nature of a test organization isalso rapidly evolving. Twenty years ago, most testers werepart of the overall development group and often reported todevelopers. This limited their ability to be independentevaluators as the testing teams were under the influence ofdevelopers. In the late 1990s and early 2000s as testingstarted to become an accepted professional career, we startedto see independent test organizations co-exist withdevelopment groups. In some instances, testing organizationseven started to report to VPs of Operations in order to bridgethe gap between pre-production and productionmanagement. With the advent of agile delivery, we expect tosee an increased trend that moves away from clearly defineddevelopment and testing organizations to a project teamconcept that combines the best talents from development,testing and business teams, working together towards acommon business goal.

Development skills (C++, Java,etc)

Scripting skills (Perl, Python, etc)

Database skills

Mainframe skills

Networking skills

Security skills

Business knowledge

QA skills

Q: When hiring testers, which of the following skills are most important to you?

Figure 5: Companies value business domain knowledge in testers above development skills. }

12

Outsourcing continues to play a key role in qualitymanagement and, by all indications, this trend will increaseover the next few years. More companies are choosing tofocus on their core business and outsource functions like QA to third-party vendors.

Europe is leading the way in outsourcing testing projectswith 65% of European respondents confirming their use ofoffshore testing. In Asia however, an overwhelming 85% ofrespondents say that they do not currently use outsourcingand are instead relying on internal resources for their testingactivities. This low level of interest in outsourcing is mostlikely attributed to the fact that labor costs in Asia are stillsignificantly lower than in Europe or in North America. (See Figure 6.)

Several key trends are emerging in outsourcing:

1. QA outsourcing is on the rise, but companies preferdomestic or near-shore vendors

Of the respondents who outsource, over half say that theyare increasing the use of offshore QA resources, with another29% planning to keep their outsourcing at current levels.Only 19% of respondents are planning to reduce the numberof outsourced testing projects. Not surprisingly, the largestnumber of responses for reducing outsourcing investmentcomes from Asia and the smallest from Europe. In fact, 41%of European respondents say that their organizations areincreasing the use of outsourced resources for testing,compared to 30% in North America.

Over a third of European companies (35%) prefer their QAvendors to be co-located with their company employees,26% prefer to outsource to India and 22% to near-shorelocations. (See Figure 7.) Most European countries – such asthe United Kingdom (UK) – have favorable laws that allowthem to use affordable long-term local or near-shorecontractors. Many of the organizations that we researchedhave QA organizations completely staffed by long-termcontractors (with 20 continuous years or more in somecases) and niche local consultants. This would not bepossible in the United States (US), where independentcontractors are traditionally more expensive than outsourcinglocations. Furthermore, large enterprise accounts in the UShave internal policies that limit the amount of time aresource can be employed as a contractor before the contractmust be converted into a full-time employee.

Other locations

Near shore within my country or continent

India

Prefer contractors co-located with my employees

EuropeNorth America

1%

Yes

No

North AmericaEurope

AsiaROW

Q: Does your organization use outsourced resources for testing?

Figure 6: Europe leads the way in QA outsourcing trend, while Asia shows thelowest percentage – possibly because of lower labor costs in the region. } Q: What would be your ideal location to outsource your

testing activities?

Figure 7: Companies prefer to have their QA providers close to their employees, followed by outsourcing to India and near-shore locations. }

13

Naturally, when QA service providers are co-located with thecompany’s employees or are close in language and culture,the flow of information and communication is much easier.The exception to this trend is outsourcing to India, but webelieve it can be explained by India’s leadership andexperience in providing QA services, cost advantages anddeveloped infrastructure that supports communication andcollaboration.

2. Outsourced resources are becoming more skilled and business-oriented

Similar to the trend that we saw in internal QA hiring,companies are beginning to place a higher premium onbusiness and domain knowledge when selecting theiroutsourced service providers. One of the biggest challenges isthat vendors do not provide enough skilled resources withthe business and domain knowledge of the client. Only 14%of all survey respondents say that their outsourced resourcespossess relevant domain and business knowledge.

In the earlier phases of outsourcing, companies were seekingcheaper resources to augment their capabilities at lower cost.With rates at overseas outsourcing centers increasing,companies in North America and Europe are re-evaluatingprofiles. In the near future, companies located in the Westernhemisphere will require outsourcing resources to providedomain intellectual property (IP) knowledge, industry-specific experience and highly technical testing skill sets thatare backed by robust best practices. This shift in demand hasresulted in a rise of many vertical-specific service providersand has forced other service providers to further organizetheir solutions into verticals.

3. Automation could reduce the number of outsourced resources

Companies in Europe and North America that useoutsourcing without proper governance often find that theiroutsourced QA teams grow uncontrollably large as a result ofhaving to add more people to solve quality problems. Thistype of unchecked growth, coupled with rising costs ofoffshore testing services, threatens to negate any cost-savingbenefits achieved by outsourcing. In the survey, mostrespondents say that their organizations can reduce thenumber of outsourced testing resources by increasing theirlevels of automation. With more QA automation and ahigher-skilled labor force, companies would feel comfortablereducing the size of their outsourced QA teams and projects.

4. New value-added services demanded from outsourcing partners

There is a divide between organizations that are looking tocontinue business-as-usual by simply augmenting their QAteams with offshore resources and organizations that arelooking for more value-added services from their outsourcingvendors. The more mature organizations are now looking togain more specialized services, such as industry-specificquality solutions with pre-built requirements, test cases orautomation from outsourcing vendors. On average, 45% ofrespondents say that they would increase outsourced testingresources if they could have more value-add from vendors. Afurther 21% say that, while they would not increaseoutsourced resources, they would still be interested invendors’ solutions and IP. Similarly, 36% of respondents saidthat they would use outsourced testing services if theyoffered expertise and methodology in testing-specificapplications such as SAP. This clearly indicates that in orderto continue the successful relationship with their clients,outsourcing vendors need to offer more specialized, value-added services, and not just be an extension of the client’sworkforce.

Over a third of Europeancompanies prefer their QAvendors to be co-located withtheir company employees.


14

Agile development methodology took the IT world by stormwhen it first emerged nearly 10 years ago. It was a dream-come-true for developers who no longer had to spend weeksgathering project documentation, and instead could focus onwhat they liked best, writing code. However, mostimportantly, it held a big promise for business and ITexecutives who were tired of taking on huge projects only tosee them fail, run over budget, miss deadlines and deliverlittle or no business results. Companies big and small beganturning from waterfall and other traditional developmentmethods to agile, hoping to improve businessresponsiveness, make their applications more adaptable tochanging market conditions and enhance the quality of theirIT systems.

Nevertheless, the road to agile wasn’t always easy. Somecompanies tried to switch all their major developmentprojects to agile only to find out that their IT and QAorganizations did not have the required skill set to supportthe new development methods. Others, concluding thatfaster development means that formal quality managementprocesses are no longer needed, attempted to assign all QAfunctions to Research & Development (R&D) engineers,negatively impacting the quality of the finished product. Stillothers tried to force agile on large, globally distributedteams, without taking into consideration the high degree ofcommunication and collaboration required to work in anagile environment.

Despite many challenges, agile is here to stay, and it isgaining popularity. Our survey and other industry studiesindicate that over 60% of all companies have either alreadyadopted agile methods or they are planning to do so in thenear future. Companies of all sizes, across all verticals andgeographies, are adopting agile for both new and existingprojects to give a competitive edge to their business. (See Figure 8.)

There is also a correlation between the types of IT projectsthat companies continue to invest in and the growingpopularity of agile delivery methods. Most companies arefocusing their IT investments on projects that generateimmediate ROI, and agile is ideally suited to deliverfunctional applications fast and with predictable quality.

By Ken Brennock, Manager, Agile and Automation Practice, Sogeti, and Brad Hipps, Senior Manager, Applications Solution Marketing, HP

North AmericaEurope

Asia ROW

Yes

No

(A)

New development

Large projects

Web technologies

Client server technologies

Maintenance work

Small projects

Mainframe/legacy technologies

For non-IT projects

(B)

Q(A): Do you leverage agile development/delivery methodsin your company’s ALM?

Figure 8: Companies in all industries leverage agile methodsacross projects and technologies.

Source: A commissioned study conducted byForrester Consulting on behalf of HP, April 2010.Base: 113 IT decision makers with insight intobudgeted modernization activities currently usingagile development approach.

}

Q(B): Where in your organization is agile developmentbeing applied? (select all that apply.)

15

Agile can helpsignificantly improveapplication quality,productivity and time-to-market, but it isnot a cure-all remedy.


the way we see it

16

Best practices adopted by successful agile companies include:

1. Set realistic expectationsAgile can help significantly improve application quality,productivity and time-to-market, but it is not a cure-allremedy. Perhaps the most significant benefit of agile is that itprovides a much more realistic measure of progress. Insteadof gauging the effort, developers are measuring true progress,which brings them closer to the business and makes themmore aligned with the stakeholders’ expectations.

Survey respondents say that time-to-market is the greatestbenefit that their organizations realized by adopting agile.Agile is helping IT teams deliver applications on time andwithin budget while minimizing rework due to its iterativeand predictable nature, greater productivity and fasterreaction to change.

2. Do not ignore qualityMany companies claim that they are moving away fromwaterfall because they are discovering issues too late in theapplication development process when it is too costly to doanything about them. Unfortunately when they move toagile, they find themselves falling into the same routine –develop in fast sprints, but still test at the end of an iteration,or even after several iterations.

Quality should be part of every step of any agiledevelopment process. Each sprint should be composed ofsome amount of development and testing. This is the onlyway to identify issues and project risks earlier in the lifecycle.With the right approach, agile organizations can achievesignificant improvements in quality. In fact, surveyedcompanies perceive quality improvements as the secondbiggest benefit of agile – behind only time-to-marketadvancement. (See Figure 9.)

3. Find the right skill setAgile brings development and testing functions closertogether, but it does not automatically turn developers intogood testers, or make testers more familiar with thedevelopment process. Organizations need to take a closerlook at the skill set required by the QA team to effectivelysupport agile methods.

Agile testers need to be more versatile than traditionaltesters. On the one hand, they must be more technical, morefamiliar with development practices, and comfortable withusing non-traditional test automation tools to validateGraphical User Interface (GUI)-less applications. On theother hand, they need to be close to the business tounderstand the requirements, work with end-usersthroughout the project, react quickly to change and tieapplication quality directly to business value. Because agileteams are typically small (six to twelve people) or acollection of smaller teams (“scrum of scrums”), an agiletester can be compared to someone working in a smallcompany where everyone wears multiple hats. With the rightskill set and a strong understanding of all aspects of thebusiness, development and testing, QA can truly make agilean effective and successful method for the company.

4. Emphasize communicationSome agile “purists” may say that companies cannot haveagile when they have distributed teams. In contrast, thesurvey results confirm that the reality of today’s business isdistributed teams, and many of them are successfullyadopting agile methods. Nearly half (46%) of all respondentsleverage offshore resources in their agile delivery. Theseresults clearly indicate that it is possible to be distributedand be agile. The key is to stay connected. The two biggestchallenges when working on agile projects with outsourcedpartners are poor communication and lack of mutuallyaccepted collaboration technologies. (See Figure 10.) It is ourexperience that in offshore situations, companies mustemphasize the management of cultural and languagedifferences since they often directly contribute tocommunication problems.

Through proper work planning, team structure, central assetrepositories and open communication, distributed teams canachieve agile success.

Cost savings

Quality of application

Time-to-market

Resource utilization

Q: In which areas has your organization seen an improvementas a result of moving to an agile delivery method?

Figure 9: Agile yields significant improvements for IT organizations.}

5. Achieve speed through automationAgile thrives on efficiency. Agile delivery teams need to maketheir processes as efficient as possible, and this meansincreasing the leverage of automation frameworks and testoptimization techniques such as risk-based testing andorthogonal arrays. Developing and managing requirements,generating test data, storing project assets, building andrunning tests – all tasks that are run routinely – cannot staymanual without slowing down the process and creatingunnecessary burden on the teams. Technologies andtechniques that are considered “nice to haves” in thetraditional application lifecycle have become necessaryenablers of agile success.

Traditionally, test automation in an agile environment wasdifficult due to lack of application UI. Most availableautomation tools required a GUI to start building testcomponents and scripts. However, the emergence of the newgeneration of test automation solutions allows testers tobegin automating much earlier in the process, without theneed for the UI to be completed. This significantly improvesthe speed, quality and consistency of agile testing andcontributes directly to higher application quality.

6. Organizations may need to find their own, uniqueform of agile

Agile can be more complicated than traditional developmentmethods, and the complexity is not only about writing codeand testing at a faster pace. Agile changes the very way ITinteracts with the business. Adopting agile means finding thebalance between speed and quality, responsiveness andpredictability, embracing change and economies of scale,minimal documentation and accurate reporting.

Many organizations find success in taking the best oftraditional methodologies and applying them to their agileprojects, creating a unique hybrid model that works forthem. Perhaps some testers are more used to gatheringrequirements than user stories, or specific project teams needmore granular reporting to satisfy regulatory compliance.Companies can find an agile method that’s right for them.The main principles of agile – building in small iterations,hands-on communication with the stakeholders, rigoroustesting and receptiveness to change – are going to continueto work, even if teams add elements from other methods.The end result will be higher quality applications and greaterbusiness value.

For the testing organization, agile represents both anopportunity and a challenge. The majority of surveyedorganizations (67%) are not currently using agile-specifictesting methods to support agile application delivery, whichsuggests that the role of the agile tester is still not welldefined. For years, testers have been trying to ensure thatapplication quality be considered earlier in the lifecycle.With agile, QA can help make it happen by providing theright skill set, including technical knowledge of thedevelopment process and strong business process expertise.

Agile is requiring teams to work with a greater degree ofcollaboration. It collapses the silos making the entire teamfeel ownership of the product. Testers no longer have theback-seat role where they are forced to wait until theapplication is delivered to them. They are actively engagedfrom beginning to end. Agile is a different way of working. It requires a new level of competency and commitment fromthe QA organization, because it is not agile development, butrather agile delivery that ultimately makes projects successful.

17

Agile delivery cannot be successfulleverging offshore resources

Lack of automation

Lack of collaboration technologies

Lack of communication/communication problems

We do not have any challengeswith the offsore resourcesfor agile delivery

Q: What is the biggest challenge working with offshore resourcesfor agile delivery?

Figure 10: The biggest challenge when working with outsourced partnersis lack of communication and lack of collaboration technologies.}


18

Historically, IT has planned for application capacity byallocating a dedicated physical server for each application orproject. The general rule was to run each server at about50% capacity to allow for planned and unplanned spikes inactivity without degrading performance. In suchenvironments, datacenters were expanding rapidly, and ITmanagers did not have the ability to easily reduce theapplication footprint and use existing capacity in a moreefficient manner. In reality, most servers are only utilized atabout a 15% rate, and IT doesn’t have an accurate way topredict actual capacity needs.

To minimize capital expenditures and gain better control oftheir IT resources, companies are beginning to take advantageof the delivery model known as cloud computing. In cloudcomputing, applications and information are provisioned on-demand as a shared resource. Cloud architecture can be setup as a “public” cloud – with services dynamically deliveredfrom a third-party provider hosted outside of the firewall;“private” cloud – where a cloud-like architecture can be setup over the company’s private network (inside the firewall);or a hybrid model utilizing a combination of internal andexternal providers.

Public and hybrid cloud customers typically pay for capacityand applications on-demand, enabling them to streamlinetheir IT architecture and reduce operational expenses such asdatacenter maintenance and power consumption. Our surveyfound that IT professionals who deploy their applications onthe cloud view cost reduction as the number one benefit ofswitching from the physical infrastructure. This is followed byincreased agility – giving IT the flexibility to move theworkloads around and add capacity as needed. (See Figure 11.)

If a company is planning to run a big promotion orintroduce a new, highly anticipated product, they must planfor increased traffic to their web site and higher ordervolumes coming through their IT systems. Traditionally, ITwould have had to purchase additional server capacity anddeploy it in their datacenter, requiring added staff and extramaintenance expenses. Using the cloud architecture, ITmanagers are now able to quickly replicate their applicationenvironment on the cloud and have the capacity they needon-demand. The same holds true for applications that areonly used several days out of the month – such asaccounting or financial systems. Application performanceduring those critical times is essential, but for the rest of themonth it may remain virtually idle, so capacity can be turnedoff or re-allocated elsewhere.

Companies are still cautious about moving all of theirmission-critical applications to the cloud. In our survey, only4% of respondents said that their entire applicationlandscape has been moved to the cloud, with the biggestgroup of participants (35%) indicating that up to a quarter oftheir applications are now cloud-based. (See Figure 12.) Weare seeing an increased number of companies virtualizingtheir applications as the first step towards moving to a publicor private cloud. We view hardware virtualization as anenabling technology of cloud computing. Virtualization isessentially commoditizing hardware. It enables IT todisconnect its applications and environment from thehardware and move applications quickly regardless of whatplatform they reside on. Virtualization eliminates configurationsat the hardware level by providing a layer of abstraction –paving the way for companies toward cloud computing.

By Dennis Corning, Senior Manager, Product Marketing - BTO Virtualization, HP

It is trendy

Time-to-market

Increased agility

Cost reductionNone

100%

Up to 75%

Up to 50%

Up to 25%

Q: If you use cloud, what is your company’s top reason?

Figure 11: Nearly half of IT professionals deployapplications on the cloud to save costs.

Q: What percentage of your company’s applications are virtualized by your internal IT department?

Figure 12: The majority of companies deploy a quarter of their applications on the cloud.

}}

19

Despite the Service Level Agreements (SLAs) offered bycloud vendors, moving applications to the cloud stillrepresents a major shift in architecture. Applications cannotbe deployed without thorough testing and validation. In fact,a quarter of all survey respondents indicate that theyencountered application performance issues within the firstfew months of moving to the cloud infrastructure, followedby 20% who experienced security and vulnerabilityproblems. (See Figure 13.)

We believe that within the next 12 months, a growingnumber of organizations will begin to take advantage of thecost savings and agility that cloud computing has to offer.From a testing perspective, our survey found that over halfof the respondents’ companies have already begun toleverage cloud architecture for their testing efforts, such ashosting their testing environments on the cloud or generatingload traffic for performance testing from the cloud.

The key to the successful adoption of cloud architecture is infinding the right balance between the predictability andcontrol of traditional datacenter environments and the agilityand speed of cloud-based deployments. With properprocesses and testing practices in place, organizations canminimize the risk and rationalize their IT infrastructureusing new cloud technologies.

We have issues but we do not know the reason

We have not seen any issues

Infrastructure performance problems

Application functionality issues

Security and vulnerability problems

Application performance issues

Q: If you host applications on the cloud, what are the top issues that youencounter within the first few months?

Figure 13: Just like applications deployed on a physical infrastructure, cloudapplications need to be thoroughly tested to avoid production problems.}


In�this�year’s�report,�we�selected�six�specificindustries�to�profile:�Consumer�Products�and�Retail,Distribution�and�Logistics,�Financial�Services,�PublicSector,�High�Tech�and�Telecommunications.�Eachindustry�section�contains�a�high-level�overview�ofdominant�IT�tendencies,�as�well�as�specific�findingsfrom�our�IT�professionals’�survey�on�QA�trends,�ITinvestments,�outsourcing�and�adoption�of�newtechnologies�and�methods.

Consumer Products and Retail (CPR) is one of the broadestindustries, combining food and beverage manufacturers,makers of health and beauty products, and several other sub-sectors involved in making and distributing common goodsused by consumers.

CPR companies rely heavily on enterprise resource planning(ERP) and customer relationship management (CRM) systems,as well as supply chain management applications. Somecompanies have global instances of these systems to helpmanage their product development, supply chain, sales andmarketing activities. Others have regional or divisional systems.

Traditionally, the role of IT was seen as a supporter of thebusiness functions, but not as an equal partner responsible forthe success of the overall mission. This view is changingrapidly. With increased competitive pressure, growing businessand supply chain complexity and the ever-increasing amountof information about consumer behavior and industry trends,the business is beginning to see IT as a collaborative partner.As a result, companies are placing a greater premium onaligning business and IT priorities.

In particular, the area of predicting and monitoring consumerdemand and balancing it with the supply of available productsdemonstrates the strongest synergy between business and IT.Demand for consumer products can often be volatile, and therecent downturn in the economy has shifted consumer buyingpatterns. To remain competitive, companies need to be able tocapture, understand and respond to demand data fromconsumers in a timely manner. They need sophisticated,comprehensive and integrated IT systems that can accuratelymeasure and predict demand trends, as well as tightly managethe product supply chain. A slight misbalance between supplyand demand results in either too much product in the valuechain or not enough to satisfy the customer’s appetite,sometimes resulting in empty shelves (lost sales).

Consum

er Products and Retail: Using the Downturn to Drive Improvem

ent

By Brian Girouard, Leader, Global Consumer Products & Retail, Capgemini

20

21

When asked about the greatest challenges facing theirorganizations in the application lifecycle, CPR respondentshighlighted the following three areas: getting businessconsensus on the right investment (27%), understandingbusiness requirements (27%) and getting development right(20%). Not surprisingly, when asked about the relationshipbetween business and QA, the majority of the ITprofessionals from CPR companies (63%) say the businessbelieves the only role of QA is executing test scripts. This isthe highest percentage among all industries, indicating asignificant disconnect between the business and QA.

The lack of relevant industry domain expertise among testersin the CPR sector also points to the misalignment amongdifferent organizations as well as a siloed approach toapplication quality. On average across all sectors, 10% of allrespondents say their testers have “very little” expertise, 72%have a “few years” and 18% consider themselves “truedomain experts.” By comparison, CPR respondents say that67% of their testers have little business knowledge, and nonecan be considered industry domain experts. (See Figure 14.)

The key to improving application quality is to re-examine therelationship between business and QA and make quality atrue part of the application lifecycle – from gatheringrequirements to monitoring the application in production.This approach will also help improve the relationship withoutsourcing vendors. Almost one half of CPR respondentsindicate that their companies are leveraging outsourcedresources for testing and are planning to increase their use ofoffshore vendor services in the next two years. By streamliningcommunication between business and IT and making qualitya part of every stage of the application lifecycle, companieswill be able to get better results from their outsourcingengagements and achieve greater efficiency in their quality processes.

CPR companies will need to make changes to their qualityprocesses in the near future. Their agility and competitivenessdepend on it. In addition, as many of these companiesproduce products for human consumption that are subject togovernment regulations, they need to have systems andprocesses in place to ensure that products are being handledcorrectly and can be recalled if necessary. Only integratedand reliable IT systems can truly support the entire productlifecycle – from manufacturing to sale or recall.

The recent recession has led CPR companies to focus onimproving business and operational efficiency. Organizationsare working on reducing their overall inventory andlocalizing their offerings to be more relevant for consumers.CPR companies are beginning to invest their IT dollars intogetting closer to consumers and better managing their supplychains throughout the business. The survey shows that CPRis among only three industries (the others were Media andEntertainment, and Telecommunications) that indicated anincrease in IT spending – despite the challenging economicconditions. Respondents from the CPR sector agreed thattheir companies are continuing to invest in new IT projects –and not just those generating immediate ROI.

Very little

Few years relevantindustry domain knowledge

Experts in our industry domain

Consumer Products and Retail

Distribution/Logistics

Energy, Utility and Chemicals

Financial Services

Healthcare/Life Sciences

Manufacturing

Media and Entertainment

Public Sector/Government

High Tech

Telecommunications

Q: How much relevant industry domainknowledge do testers in your organization have?

Figure 14: CPR testers have less businessknowledge compared to other industries. }


Distribution and Logistics: The Road to IT Transformation

By Dennis Wereldsma, Worldwide Transportation Sector Leader, Capgemini Consistent with the other sectors profiled in this report,

Distribution and Logistics is a diverse segment that includesa variety of logistics and transportation service providers:freight forwarders, postal service providers, the maritimesector, rail and even airlines and airports. Naturally, eachsub-sector has a very different approach to IT with adifferent set of priorities. However, in analyzing thecompanies in the Distribution and Logistics sector, wediscovered that some of the most interesting trends in IT areemerging among the logistics service providers, so we choseto focus on the freight forwarding sub-segment for a moredetailed analysis.

Freight forwarding companies rely heavily on IT systems fortheir core transportation functions, such as scheduling,planning and dispatching. However until recently, there hasnot been a standard ERP-like packaged solution on themarket to support this very specialized sector. As a result,each freight forwarding provider resorted to building its owncustom set of IT applications. Depending on the size of theprovider and the complexity of its operations, thesededicated systems often take several years to build andrequire substantial effort to maintain and keep up-to-date.

The real issue with custom systems is that they can beextremely difficult to integrate when companies’ IT systemsmerge. Logistics companies typically grow by acquisition,with larger providers absorbing smaller players and theiroperations. After several acquisitions the company’s ITlandscape becomes cluttered with redundant systems – someobsolete and others still supporting disjoined, specializedfunctions. These systems are too costly to maintain and toodifficult to integrate with newer software packages, butalmost impossible to retire. Our research shows that logisticsoperators are at a pivotal point in their IT transformationjourney. Many companies are trying to define the roadmapfor rationalizing their application landscape to achievegreater agility, security, reliability and compliance, but thechange may not be as fast as in some of the other sectors –such as Financial Services or Telecommunications. Logisticsis a low-margin industry that has traditionally invested mostof its resources in keeping its vehicles running and goodsdelivered, not advanced IT technology. Only about 2% oflogistics companies’ annual revenue is spent on IT, and over60% of all surveyed logistics providers still admit that thealignment between business and IT needs improvement(Source: The State of Logistics Outsourcing, 2009 Third-Party Logistics, Capgemini).

22

23

Nonetheless, change is coming. Seeing the potential and thegrowing market, two of the leading ERP providers – SAP andOracle – have recently entered the transportationmanagement application space. Oracle’s new transportationmanagement system is already available on the market, andSAP’s new application is expected in 2011. We believe thatmany logistics companies will take advantage of theseofferings to consolidate their processes and systems into asingle, standardized application infrastructure.

Logistics providers are also feeling the pressure from theircustomers to build a global cross-industry IT platform usingservice-oriented architecture (SOA) and open-industrystandards. This type of platform would enable logisticsproviders to streamline collaboration with customers,partners and regulatory agencies, while creating an efficientinfrastructure and data repositories. Not surprisingly,logistics companies are turning to agile developmentmethods to help them build new SOA platforms. Seventypercent of all survey respondents in the Distribution andLogistics sector say that they are leveraging agile in theirapplication development and delivery practices, and allrespondents say that they use integrated automation tools tosupport their agile initiatives. (See Figure 15.) The numberof projects where agile is used is still small – typically undera quarter of all IT projects – but we believe this indicates thebeginning of the true IT transformation in the logistics sector.

Logistics companies are often unwilling to decommission oldapplications for fear of potential disruptions to shipmenttracking, customer invoicing and virtually every other area oftheir operations. They are also not able to easily purge olddata that continues to grow uncontrollably – due tocompliance and data retention requirements. At the sametime, many companies admit that they cannot continue tooperate in this fashion, spending up to 80% of their ITbudgets supporting ever-expanding datacenters, aginginfrastructure, multiple data repositories and complex one-off integrations.

Logistics and transportation companies are also beginning tobring their applications to the cloud to reduce hardwarerequirements, improve application quality and performanceand reduce maintenance expenses. Even traditional serviceproviders, like the Postal Service, are beginning to look fornew ways to increase revenue and boost operationalefficiency. To remain competitive, they need to look beyondtheir traditional modes of operations and enter new businessarenas with quick, convenient, easy-to-use and reliablecustomer-facing applications. To bring large volumes of newsystems to the market quickly, providers are turning tovirtualization technologies that offer the advantage of fasterdevelopment times, increased quality and shorterdeployment and QA cycles.

We believe that adoption of new technologies and methods –such as agile delivery and virtualization, as well asmodernizing the application landscape and developing globalIT platforms – will fuel the future growth of logistics serviceproviders. As the economy continues to recover andcompanies increase their IT spending, we will see moreinvestment going toward strategic projects and initiatives.

Yes

No

Q: Do you leverage agile development/deliverymethods in your company’s ALM?

Figure 15: The Distribution and Logistics sector is at theforefront of adopting agile delivery methodologies.}


Financial Services: New

generation demands new

solutions

By Jim Washburn, Vice President, Banking Sector Practice Leader, Capgemini

The Financial Services industry is composed of three broadsectors: Insurance, Banking and Capital Markets. In thissection, we will focus our attention on the banking sectorand the strategic IT initiatives that are helping to revamp thebanks’ entire distribution system and change the way banksinteract with their customers.

In 2010 the world’s banks and other financial organizationsare undoubtedly still feeling the effects of the recenteconomic crisis. However, it is not the recession that ischanging the way that banks are conducting business. Theemergence of a new generation of customers is makingbanks rethink their entire distribution systems. Today’s 20-and 30-year-olds have little interest in walking into atraditional bank and talking to a teller. Unlike their parents,they rarely use call centers or balance their checkbooks. Thisnew generation of banking customers prefers to conductmost of its transactions online through its computers orincreasingly on mobile devices.

Unfortunately, today’s banking distribution system is notbuilt to support this next generation of wealth. Most banksstill offer little more than a utilitarian web presence, andunlike sophisticated online retailers, banks are not yet takingadvantage of the tremendous amounts of personalinformation and behavior patterns that they collect on theircustomers. Today’s banks are not yet able to effectively usetheir customer data to create personalized offers or suggestservices based on individual customers’ choices andpreferences.

Banks continue to invest money in new projects, with areassuch as web presence and data warehousing attracting themost IT investments. They need to be able to fullymodernize their distribution channels in the next five to tenyears to be able to keep pace with younger consumers. Thismeans transforming each transaction into an electronic saleschannel and consolidating data to better understandbehavior patterns to deliver a personalized experience on theweb and through mobile devices.

24

No

Only those that generateROI in less than six months

Yes

(B)

25

To support this transformation, banks are beginning to takeinventory of their IT assets and are taking steps towardsupdating their application landscape. Many of today’sbanking sector companies rely on custom-developed legacyapplications that are in excess of 20 years old. These systemsrequire extensive maintenance, are not well integrated andare generally not the types of engines that could support acompany’s future growth. Many financial institutions havealready begun the process of replacing these custom systemswith packaged applications, and as the economy continuesto recover, we expect to see more banks investing inapplication modernization.

Like most other sectors, banks have reduced their ITspending during the recession. However, 80% of our surveyrespondents say that their companies are continuing to investin new IT projects. This clearly shows that Financial Servicescompanies are continuing to fund their IT initiatives, but arecareful to focus their investments on the most strategicprojects – such as web presence and applicationmodernization. (See Figure 16.)

New types of applications require new delivery methods –Financial Services organizations were among the earliestadopters of agile. Through continuous collaboration with thebusiness, IT is able to deliver new business functionality inshort, frequent cycles. In our survey, over two thirds ofrespondents from the Financial Services sector indicate thattheir companies are leveraging agile in their ALM.

Financial Services organizations are also improving their ITagility through more effective resource management. Close to70% of our survey respondents say that their organizationsare using outsourced services for testing and nearly halfindicate that their reliance on outsourcing vendors isincreasing. Cost is not the only driver behind the growingdependence on outsourcing. Outsourcing can provide avariable resource for organizations to deliver services on-

demand quickly and efficiently, as well as add industry andprocess expertise. In fact, over half (56%) of surveyrespondents in the Financial Services sector suggest that theywould increase their use of outsourced vendors even furtherif they could receive more value-added services such asindustry-specific quality solutions, pre-built requirements,test cases and automation components.

In general, the banking industry continues to be optimisticabout the current state of IT investments and emergingtrends in software development and quality. FinancialServices companies – and specifically banks – are among asmall number of regulated companies where applicationperformance testing is considered a requirement. Bankingand investment banking applications require precision intesting that is not commonly necessary in other industries.With multi-trillion dollar transactions taking place dailybetween financial institutions, accuracy and precision arerequired for all IT applications. For example, all calculationsmust be tested up to seven decimal points – as fractions ofpennies add up to large amounts in the financial industry.

We believe this sector will see more technologicaladvancements in the next five years than it has experiencedin the previous ten. New technologies will emerge that willsupport the next generation of consumers. New tools anddelivery methods will continue to bring more functionality inshorter cycles. Modernized application landscapes will bringefficiency to IT applications – resulting in new, integratedservices that banks are able to provide to their customers. In the end, customers will experience a level of quality andconsistency that is higher than anything that is available today.

Increased IT investments

Stayed the same

Reduced IT investments

(A)

Q(A): How have the current economic conditionsaffected your organization’s IT investments?

Q(B): Is your organization still investing in newapplication development projects?

Figure 16: Financial Services companies are shiftingtheir investments to the most strategic IT projects.

}


Public Sector: Security, Performance, Efficiency

By Stanislas Cozon, Managing Director, Global Public Sector, Capgemini

Capgemini’s global Public Sector includes organizations infive primary areas: Tax and Welfare, Public Security,Healthcare, Regional and Local government, and Defense.This is a complex and diverse sector that encompasses bothlocal government bodies and large internationalorganizations such as the IMF, UN or NATO.

All government organizations are subject to strict rules andregulations that impact their IT practices and policies.Although these regulations are often unique to each sub-sector, all mandate that government organizations maintainsecure, efficient and cost-effective IT systems – not just forstoring, but also for analyzing and sharing information. Forinstance, healthcare organizations are required above all toprotect patient privacy. At the same time, they need tosecurely share information between hospitals and otherfacilities to help providers gather complete data on thepatients’ health, diagnosis and treatments.

Depending on their function, government agencies use avariety of technologies, from ERP and CRM packages tohighly specialized analytical applications, to help identifycriminals or calculate the amount of carbon emissions incompliance with environmental regulations. With IT budgetsshrinking and the public eye closely focused on how thegovernment spends its resources, agencies are beginning totake a more strategic approach to their investments – re-examining ROI, re-evaluating project success and optimizingtheir IT infrastructure. Most of the current investments in ITare focused on projects that promise greater cost savings andefficiency in the context of operational excellence andimproving the customer experience.

One such strategic initiative is data centralization. With strictrequirements for retention of sensitive data, many PublicSector IT organizations face the problem of data growthimbalance – datacenters are expanding faster than informationcan be removed. Agencies are working on creating a clearstrategy for controlling data growth, securely archivingsensitive information at a central location and building datagrowth control into their IT operations and processes.

Another initiative that is aimed at increasing efficiency andreducing costs is moving applications to the cloud. Althoughthe percentage of applications that are being tested throughthe cloud are still relatively small, all of our governmentsector survey respondents confirm that they intend to bemoving some of their applications to the cloud in the nextthree years.

26

27

Quality is of particular importance to governmentorganizations. While a poorly performing or temporarilyunavailable application may cause a private sector companyto lose revenue or customer satisfaction, a similar problemcan be disastrous for a government agency that deals withcrime prevention, tax collection, healthcare, immigration andmany other aspects of public service and security. Equallyimportant is being able to maintain the citizens’ trust in thegovernment’s ability to safeguard personal information andsensitive data. Not surprisingly, most government organizationshave a well-defined ALM methodology that is beingconsistently followed in the majority of development andtesting projects. The government sector survey participantsalso have the highest number of responses for following astandard methodology: TMMI, TMap® and Six Sigma.

Perhaps it is the stringent adherence to traditional qualitymethodologies and the complex, highly regulated nature ofgovernment applications that makes Public Sector organizationsslow to adopt agile methods. Our survey shows that publicagencies are behind all other industries in leveraging agiledelivery methods in their ALM. (See Figure 17.) Nearly 69%of respondents in the Public Sector say that their organizationsdo not use agile methods, and among the remaining 31%,agile is being used in less than 10% of all applications.

Traditionally, government organizations did notoffshore IT or QA functions. Perhaps once again it isthe highly sensitive and secure nature of governmentIT applications that prevented public agencies fromforming technology partnerships with outsourcedvendors. However, recent quests for efficiency areinfluencing many Public Sector organizations to re-evaluate their approach to offshoring. Agencies of alltypes and sizes are finding ways to separate their corefunctions – such as social services or police protection– from supporting IT functions to gain efficiency andeffectiveness. If a local permit office has quick accessto the database containing relevant information aboutzoning and the most up-to-date requirements, theywill better service their clients. They may evenconsider opening a self-service portal for customers to find information and apply for permits online.

Not surprisingly, Public Sector organizations are slowlychanging their attitude toward outsourcing. Nearly a third(32%) of survey respondents say that their agencies areleveraging QA outsourcing, and 21% suggest that they planto increase the use of outsourced resources in the nearfuture. Similar to private sector companies, governmentagencies are looking for more value-add services from third-party providers. A third of all survey respondents say thatthey would be more likely to use outsourced testingresources if they delivered more value-added services such asindustry-specific knowledge or offered expertise andmethodology in testing specific applications such as SAP.

Rising costs, an aging population, drops in governmentrevenues due to slow economic recovery and growingcitizens’ expectations for more personalized services areforcing government agencies to start reshaping their ITsolutions. The future of government IT lies in improvedefficiency, agility, reduced administration costs, improvedquality and security. We believe that government agencieswill continue to take advantage of the new technologies suchas agile and virtualization, and improve efficiency throughoutsourcing and application modernization.

Consumer Productsand Retail


Energy, Utilityand Chemicals

Financial Services


Manufacturing



High Tech

Telecommunications


Q: Do you leverage agile development/deliverymethods in your company’s ALM?

Figure 17: Government agencies are slow toadopt agile. }

High Tech: Quality as Com

petitive Success Factor

By Guido Kamann, Vice President, Global High Tech Sector, Capgemini

Capgemini�s High Tech sector combines a diverse group ofcompanies involved in computers and peripherals, consumerelectronics, printing and imaging, telecommunicationsequipment, software and services and semiconductors.

High Tech companies have been among the fastest growingbusinesses around the world, fueled by insatiable consumerappetite for faster, more powerful computers, smarter mobilephones and the latest electronic gadgets. Traditionally,technology companies have not been in the forefront ofquality. The slogan “it’s better to be first than to be better”aptly implies that software developers and equipmentmanufacturers rush to get their latest products to market tobeat the competition without much regard for quality.Consumers have been generous and forgiving. They haveaccepted that they may have to wait for the second or thirdrelease of a new product or for a service pack for the newsoftware package before they can really trust it with theirpersonal data or business processes.

As technology continues to advance and competition heatsup, the focus of technology companies is beginning to shifttoward quality. Today’s High Tech companies are working aspart of elaborate ecosystems of suppliers, partners, serviceproviders, distributors and other essential functions. For thiscomplex chain to work, participating companies have toprovide the highest quality products, parts or services.Otherwise, the relationship and trust between partners willbe broken, and the delicate system will collapse.

Perhaps even more important is the fact that consumers arebeginning to perceive product quality as a competitivedifferentiator. For high tech devices, quality is not simplyabout having a product that works as expected and doesn’thave any obvious problems. With so many products tochoose from – many featuring similar capabilities –customers see quality as the combination of usability, designand the overall user experience. When asked to compareone smart phone model to the next, customers often talkabout their preferred product’s “quality” – ease of use,breadth of coverage or the variety of available applicationsand plug-ins. This clearly shows that quality in thetechnology sector has truly become a competitive advantage.Companies that deliver the most positive user experiencecan forge ahead.

28

29

Our survey respondents concur that application quality hasimproved over the last two years, although the amount of ITspending has been reduced. Technology companies achievehigher quality with limited resources through efficiency andinnovation. They were among the first to adopt agile deliverymethods, and continue to be on the forefront of agileadoption. Almost 80% of survey respondents indicate thattheir companies use agile. Of the respondents using agile,36% of participants say that over half of their applicationsare being developed by leveraging agile methods. (See Figure 18.)

High Tech companies were also among the first to embracethe concept of moving their applications to the cloud to freeup datacenter resources and optimize their IT infrastructure.They remain forward-looking, planning to test 25% or more oftheir applications through the cloud in the next three years.

Technology companies are not as open to outsourcing oftesting services as some of the other sectors. In our survey,they were behind only Distribution and Logistics and PublicSector organizations in preferring to keep their testingfunction in-house rather than leverage offshore resources.This is not unexpected, since the reason for outsourcing is tofocus attention on the core business while outsourcingsupporting functions such as QA. For technology companies,developing and testing applications and products is the corebusiness, so not surprisingly they prefer to keep qualitycloser to the development process. High Tech has the highestnumber of respondents (76%) who say that their businessdoes not see the primary function of QA as simply executingtest scripts. With so many technology companies followingstandard QA methodologies like TMap® and TMMI, theymake QA a true partner in their application delivery process.

Greater than 75%

Up to 75%

Up to 50%

Up to 25%

Up to 10%


Q: What percentage of your applicationdelivery projects leverage agile methodology?

Figure 18: High Tech companies remain in theforefront of agile adoption.}

Telecommunications: Continuing to Invest in IT

By Greg Jacobsen, Group Director, Telecommunications Sector, Capgemini

For the purposes of this report, we groupedtelecommunications providers and network equipmentmanufacturers into the Telecommunications sector.

The Telecommunications sector is very diverse, yet manycompanies share similar trends:

n Most companies in the Telecommunications sector aremature, global providers

n They rely on IT to manage their core functions such asbilling, customer relations and business intelligence

n Despite the economic downturn, these companies continueto invest in IT, making it their second largest capitalexpense

n Most companies in this sector – specifically telecomoperators and network equipment providers – are subjectto strict government regulations, significantly affecting howthey retain and store data and deal with the customers’private information

All of the leading global telecom operators are strivingtowards moving their customer interaction entirely online inthe next three to five years. Once the self-service systems areimplemented, customers should be able to change theirphone service, pay their balance or add features withouthaving to call or e-mail the help desk. This initiative isfueling substantial investments into the development of self-service portals and integrating them into the companies’ ERPand billing systems. Our survey results confirm thatTelecommunications companies are continuing to heavilyinvest in new IT projects. Close to 76% of respondents inthe Telecommunications sector said that their companies arestill investing in new application development, and 12% saidthat their IT resources are focusing on applications thatgenerate immediate ROI.

The other emerging trend in the Telecommunications spaceis the move to consolidate, centralize and simplify thecompanies’ strategic IT systems. Rather than rely on acollection of custom, loosely-integrated applications, telecomoperators are selecting one preferred provider for eachoperations domain (ERP, billing, CRM, business intelligenceand online self-service portals) and implementing it with aslittle customization as possible. Standardizing on a singlesystem for each domain provides a significant cost advantageand reduces maintenance expenses. Most importantly, itgreatly improves application quality and reliability – whenliterally hundreds of custom-built systems are replaced withmodernized, off-the-shelf packages.

30

31

The change may not come easy to many users who areaccustomed to working with the old systems. Additionalchallenges come from the global nature of many largetelecom operators – where regional providers weretraditionally free to select and implement their own ITsystems. However, the trend is clear – most large operatorsare moving away from one-off systems or heavily customizedpackaged applications in favor of efficient implementations,streamlined processes throughout the organization,consistent user experience and standardized quality metrics.

With consolidated systems and fewer customizations, it isbecoming easier to gather requirements and track changes.In the past, it used to take six to twelve months for aworking group to capture requirements for a new system orupgrade – only to find out at the end of the process thatthese requirements have become outdated and no longermeet the needs of the business users. The new approach isbased on off-the-shelf capabilities of selected packagedapplications and minimal modifications to support thecompany’s most essential business processes.

The consolidation movement seems to come at a good timefor the Telecommunications industry. Nearly 89% of oursurvey respondents in the telecom vertical agree thatapplication complexity is increasing and 69% admit that thegreatest challenges for their IT organizations are capturingbusiness requirements and managing change.

Overall, the survey found that telecom QA professionals areoptimistic about their applications’ quality (81% believe thatquality has improved over the past two years), and they areamong the most consistent followers of testing processesacross projects (83% indicate that testing processes arefollowed in more than half of all IT initiatives).

It is the mature, consistent and repeatable approach toquality that makes outsourcing a success for many telecomcompanies. Over 40% of survey respondents indicate thattheir companies are increasing the number of outsourcedtesting projects – the third highest percentage surpassed onlyby Financial Services and Consumer Products and Retail.Interestingly, 50% of respondents say that they would furtherincrease the volume of their outsourced projects if theirvendors offered specialized expertise and methodology fortesting applications such as SAP, which is consistent with themove towards application consolidation.

Telecommunications is also in the forefront of adoptingvirtualization technologies. All survey participants in thissector indicate that they are planning to test their applicationsthrough the cloud in the next three years. Of this group,33% indicate that they expect to test over half of theirapplications through the cloud. (See Figure 19.) This is thehighest number among all verticals, and clearly indicates thissector’s commitment to technology, application quality andinnovation. In many cases, telecom operators are acting asproviders of cloud services allowing their customers to saveon infrastructure costs by deploying applications on theirproviders’ clouds.

Similar to the trend we saw in the High Tech sector, fortelecom operators, quality means more than just provingcertain product features or ensuring that their service isworking without interruption. To stay competitive and meetthe demands of the highly discriminating consumer,telecommunications companies need to provide a completeuser experience including security, speed, service and featureset. We believe that this sector is very well positioned to takeadvantage of the latest technologies and trends to continueits path of innovation, efficiency and modernization.

Up to 10%

Up to 25%

Up to 50%

Greater than 50%


Q: What percentage of your applications do you expectto test through the cloud in the next three years?

Figure 19: Telecommunications companies are committedto cloud-based testing.}

32

2010 is showing promising signs of economic recovery. Evenas IT spending begins to rise, companies are not likely to fullyreturn to their pre-recession practices. The recessionary yearshave left a long-lasting effect on the economy and havefundamentally re-shaped many current and future quality trends.

Businesses are using the downturn to take an inventory oftheir IT portfolios and re-evaluate their spending patterns.They are finding that a large percentage of their resourceswere being spent on “keeping the lights on” activities –maintaining legacy applications that no longer supportcritical business processes – rather than investing in strategicprojects that promote innovation and growth. Companies arealso closely examining their IT infrastructure to identify waysto reduce maintenance expenses and increase efficiency.

Furthermore, companies are also changing their approach tooutsourcing. They are no longer looking to simply augmenttheir workforce, but are expecting more value-add from theiroutsourcing partners. Additionally, across all industries,companies are making a shift in the types of projects theyinvest in, focusing on initiatives that provide quick ROI. Tosupport this change, they are beginning to change theirapplication delivery processes in favor of methods thatproduce higher quality applications faster. We believe thatthe rapid advancement of cloud technologies and agiledelivery methods are in no small part the result of the recentrecessions and the companies’ quest for efficiency in thetough economic climate.

Hosting applications on a cloud helps reduceinfrastructure costs As discussed earlier in this report, virtualization technologiesare helping companies increase the efficiency of their ITinfrastructure by hosting applications on the cloud. They canincrease application capacity and plan for peak loads withoutincreasing the size of their datacenters and associatedmaintenance costs. Survey data indicates that across allindustries and geographies, companies are leveraging cloudtesting and are planning to increase the number of hostedapplications in the next three years.

Agile delivery methods help businesses focus onprojects that bring fast ROIDespite the challenging economic times, companies continueto invest in new application development projects. However,the focus is shifting from complex multi-year systems toquick and nimble applications aimed at improving thecompany’s competitive edge and generating quick ROI.Traditional application development methods cannot supportbuilding, testing and deploying an application in six months.However, new agile methods can prove project ROI muchearlier in the application lifecycle and allow companies toquickly adapt to changing market needs, while reducingwaste. Naturally, a growing number of companies across allindustries are beginning to adopt agile methods – at least fora portion of their projects. (See Figure 20.) Although agiledelivery practices have been around for over 10 years, theeconomic downturn and the need to focus on faster returnsis renewing interest in agile and fueling its rapid adoption.

Greater than 50%

Up to 50%

Up to 10%

Consumer Products and Retail


Energy, Utility and Chemicals

Financial Services


Manufacturing



High Tech

Telecommunications

Q: What percentage of your application delivery projects leverage agile methodology?

Figure 20: Companies across all industries are adopting agile delivery methods.

}

33

Businesses are using thedownturn to take an inventory oftheir IT portfolios and re-evaluatetheir spending patterns.


34

Summary�•

The 2010-2011 World Quality Report offers both aquantitative and qualitative view of the emerging trends inQA testing and a forecast of how these trends will shapeALM in the future. Companies have historically invested inthe defect tracking and test automation components of ALM. We expect to see investments shift towards requirementsvisualization, business flow modeling and test environmentand data management. However, in order to secure fundsand commitment for future investments, organizations needa well-defined and accepted process for measuring ROI aswell as a complete implementation, integration and adoption strategy.

IT organizations are more consistently using ALM and testingmethodologies, but their practices are still mostly proprietary,not based on industry standards. Many organizationsdevelop and document their own best practices, which maynot afford them the full benefits and efficiencies offered by amore standard, widely accepted methodology.

The profile of a tester is changing rapidly. Most companiesare now looking for testers who have both strong technicalskills and relevant domain and business knowledge. In addition to being knowledgeable about automationtechnologies, testers need to be able to understand businessrequirements and create test validation for specific business needs.

Despite the recent recession, companies are still investing innew IT projects. Many industries are using the downturn tore-evaluate their IT spending and focus on the most strategicprojects – such as standardizing their IT systems,modernizing their application infrastructure, improvingapplication quality and re-tooling their applications to servethe new generation of customers. Many companies stillprefer to focus their resources on projects that generate fastROI. As a result, agile delivery methods are gainingpopularity across all industries and geographies. This survey– along with other industry studies – found that over 60% ofall companies have either already adopted agile methods orare planning to do so in the near future. Another emergingtechnology trend that helps companies improve IT efficiencyis cloud computing. Companies move their applications tothe cloud to be more agile, to better plan for capacity and toreduce datacenter maintenance expenses. IT organizations

are also beginning to use cloud technology for applicationtesting – for hosting testing tools on the cloud, generatingtraffic for performance testing or building up the testenvironment using cloud services.

Outsourcing continues to play a key role in qualitymanagement and, by all indications, this trend will increaseover the next few years. The majority of surveyed ITprofessionals prefer their QA vendors to be co-located withtheir company employees where they can have morestreamlined communication and collaboration. This is alsotrue for agile projects – communication is the keycomponent to successfully outsource agile project testing.With the right processes in place, almost half of allcompanies are leveraging offshore resources for their agileprojects today. Additionally, organizations are now lookingfor their outsourced vendors to provide more specialized,value-added services and assets – not just an extension of thecompany’s workforce. Similar to preferences for in-housetesters, companies are looking for their vendors to have abetter understanding of the business domain. While theeconomic climate is improving, application quality trendsand practices are not likely to fully return to their pre-recession state. Companies are using the downturn to re-tooland re-focus and will emerge with renewed commitment toapplication quality, agility and efficiency.

35

The World Quality Report Survey was sent to over 30,000CXOs, IT directors, QA managers and engineers at variouscompanies around the world. All major industries arerepresented in the survey: Consumer Products and Retail,Distribution and Logistics, Energy, Utility and Chemicals,Financial Services, Healthcare/Life Sciences, High Tech,Manufacturing, Media and Entertainment, Public Sector, andTelecommunications.

The survey also has a good distribution of company sizes;38% of respondents are employed by small companies (lessthan 1,000 employees) followed by 27% by enterprisecompanies (more than 20,000 employees), 18% by mid-sizecompanies (1,000 to 4,999 employees), and 17% by largecompanies (5,000 to 20,000 employees). (See Figure 21.)

While over half of respondents come from North America(52%), other regions are well represented: Europe (30%),Asia (9%) and Rest of the World (9%). (See Figure 22.)

In addition to the responses collected from surveyparticipants, the content of this report is supported byCapgemini Group’s benchmark data carried out on clientprojects, best practices, and over 40 years of experience intesting and quality management.

Throughout the years, Capgemini and Sogeti have developeda range of best practices, including TMap® and TPI®, whichare now accepted as industry standards around the world.TMap® is the Group’s business-driven, risk-basedmethodology for structured software testing that is designedto address the key issues of quality, time and cost – acrossthe whole development lifecycle of solution delivery. TPI® isthe Group’s model for the improvement of testing that offersinsight into the maturity of the current test process andidentifies improvement actions to accomplish the desired testmaturity level.

About�the�Study�•

Figure 21: Survey respondents by size of company }

Figure 22: Survey respondents by geographies}

Enterprise

Large

Mid Size

Small

North America

Europe

Asia

ROW


The Capgemini Group isone of the world's foremost

providers of consulting, technology andoutsourcing services, enabling its clientsto transform and perform through theuse of technologies. Present in over 30countries, the Capgemini Group reported2009 global revenues of EUR 8.4 billionand employs over 90,000 peopleworldwide. Sogeti, its wholly-ownedsubsidiary, is a leading provider of localprofessional services, bringing togethermore than 20,000 professionals in 15countries and is present in over 200locations in Europe, the US and India.

Together, Capgemini and Sogeti havedeveloped innovative, business-drivenquality assurance (QA) and testingservices, combining best-in-breed testingmethodologies (TMap® and TPI®) and theglobal delivery model, Rightshore®, tohelp organizations achieve their testingand QA goals. Capgemini and Sogetihave one of the largest dedicated testingpractices in the world, with over 6,400test professionals and a further 11,000application specialists, notably through acommon center of excellence with testingspecialists developed in India.

More information is available atwww.capgemini.com/testingwww.sogeti.com/testing

��About�Capgemini�and�Sogeti

About�Us�•

About�HPHP creates new possibilities fortechnology to have a meaningfulimpact on people, businesses,governments and society. The world’slargest technology company, HP bringstogether a portfolio that spans printing,personal computing, software, servicesand IT infrastructure to solve customerproblems.

More information about HP is availableat http://ww.hp.com

Rightshore9 is a registered trademark belonging to Capgemini.

TMap9, TMap9 NEXT, TPI9 and TPI9 NEXT are registered trademarks of Sogeti.

Copyright© 2010 Capgemini and Sogeti. All rights reserved. No part of this document may be modified,deleted or expanded by any process or means without prior written permission from Capgemini.

ContactsWe value your comments and ideas. Wewelcome you to contact us in relation to anyquestions you might have concerning the 2010-2011 World Quality Report.

CapgeminiMurat AksuGlobal Head of HP Software [email protected]

Charlie LiVice President, Global Testing [email protected]

HPErwin Anderson-SmithGlobal Alliance [email protected]

SogetiStefan GerstnerVice President, Global Testing [email protected]

Marc ValkierGlobal Partner Manager of Sogeti for HP [email protected]

Thank�YouThe authors would like to thank the team of collaborators who helped create this report: The global leaders and subject matter experts who provided insight into their areas of expertise: KenBrennock, Dennis Corning, Stanislas Cozon, Brian Girouard, Brad Hipps, Greg Jacobsen, GuidoKamann, Jim Washburn, Dennis Wereldsma; the project management, analysis, writing, and compilingof report findings: Murat Aksu, Irina Carrel, Grace Chan; the design, layout and proofing: KristinMorris, Mike Pini; and the marketing and distribution of the report: Clare Argent, Mary Johnson,Virginie Regis.

We would like to extend a special thanks to all of the individuals who participated in our World QualityReport survey and to Requirements.net for their support in distributing the survey.

Documents

world quality - Sogeti UK | Software Testing Services ... · The 2010-2011 World Quality Report is a result of ongoing collaboration between Capgemini Group and HP Software & Solutions