Performance Testing Helping It Help The Business From AppLabs

AppLabs confidential and restricted circulation 1

18th June 2008

Performance Testing Helping IT Help The Business


Performance Failures = Lost Revenue

» Yahoo News: Fri Jun 6, 2008 4:50 PM ET» Online retailer Amazon Inc said its U.S. website was

down for about 2 hours on Friday and it was unclear when service would be restored in full…

» Amazon shares fell 4.59 percent to close at $80.63

» MSNBC News: Monday March 24, 2008» NetFlix suffers 11 hour outage of its Retail as well as the

shipping and distribution Websites…» Affected customers compensated $5 each due to

downtime

» Internet Business News: July 27, 2006» Australia's Jetstar airline Website crashed yesterday (26

July) as travelers went online to take advantage of promotional fares for the carrier's new international services.


Performance Failures

» And Remember those high profile sites that failed to handle customer load during peak events:

» Victoria Secret Fashion Show…» The FIFA World Cup Soccer website…

» CNET News.com: Jan 4 2001» After a day of lengthy 11 hour outages, eBay said

Thursday that it has restored its auction system and intends to upgrade its hardware in the coming weeks to provide improved service.

» Outages of eBay's Web site attract attention because the vendor is so popular, but the problem occurs industry-wide--and not just with consumer sites.

» More hardware won't solve Web site outages -- Donna Scott, Gartner Analyst


Reluctance for Requirements

» The reluctance to defining performance requirements usually comes from:

» A lack of understanding for the purpose of performance requirements

» Time constraints due to tight and unrealistic deadlines – “market driven” and not “realism driven”

» Limited budgets - We have so many other things to worry about – it will be fine

» Ownership of systems is not understood – Key Stakeholders take a back seat when then they should be driving!


Performance Goals and their importance

“Projects without clear goals will not achieve their goals clearly”

Tom Gilb -- The Tom Gilb Principle

» Recent studies have attributed many application’s poor

performance to a lack of clear performance goals when the software was designed and developed. Further analysis has shown that much of this is caused by non-existent or incomplete performance specifications


Achieving Performance Goals

» Today’s discussion is intended to address 3 main points:

» Describe considerations for a systems performance

» Break down performance goals into tasks

» Emphasize the importance of making performance goals part of the IT software requirements definition


Planning Performance Goals

» When Planning for Performance Requirements:

» Consumer Requirements: It is critical to understand who or what will be using the Application

» Data: It is important to understand the types and amounts of data that are anticipated or required to meet business needs

» Traffic: It is important to understand the rates at which the data will need to be transacted in order to meet business goals

» Business: It is important to understand from a business perspective how much business needs conducting to make it a success – this will ultimately drive change


Performance Requirements – Roles

» Planning Requirements Phases and Responsibilities:

» The business analysts and owners should be defining the performance requirements

» The technologists and developers will be designing and developing the application to meet the requirements

» The testers will be evaluating that the application does effectively meet the requirements

» The business and its stakeholders MUST sign the application off as part of the implementation process


Performance Requirements

» Ingredients for Performance Requirements:» Define requirements in order of priority» Keep the performance requirements within the

context of the organization's business» Align the performance goals with specific

business goals» Create performance goals that can be measured» Create performance goals that are inline with

industry standards» Determine goal achievement criteria» Decide which goals will have pass fail grades and

which will have varying degrees of tolerance

» Keep in mind the business that needs to be conducted within the application to meet the consumer needs or meet the financial targets


Inter-Team Communication

» Objectives may have been set and requirements defined but sometimes the business needs change

» The key to successfully implementing performance testing is continual communication among team members – manage change accordingly

» An effective mechanism of communication and collaboration will allow the team to make good performance related decisions, whilst changes are easy to make

» The methods you use to communicate plans,

priorities and changes are not that important as long as you are able to adapt those changes without requiring significant re-work


Anatomy of a Performance Test

» Requirements

» Discovery

» Test Planning

» Automation

» Testing

» Measurements

» Analysis

» Tuning and Re-Testing


Conclusion

» Costs associated with lack of testing and understand application performance are evident

» Performance Requirements need to be oriented around business goals – its usually the business who wants and owns the system

» The roles of performance testing» Business» Development» Testing

“If all these are successful, you can be sure that IT really can help the

Business"

Technology

Performance Testing Helping It Help The Business From AppLabs