The Performance mission is to ensure a positive user experience by monitoring and eliminating system bottlenecks, establish response time baselines for trending analysis, validate capacity expectations and ensure application compliance with defined Service Level Agreements (SLAs).

Mission Statement

Deliverables ensure accountability of the application baselines, application trending and tuning results to validate capacity expectations and ensure application compliance with defined SLAs.

Performance Deliverables

All organizations complete one or more performance tests to determine how fast some aspect of a system performs under a particular workload. It can also serve to validate and verify other quality attributes of the system, such as scalability, reliability and resource usage.

Why - Performance Engineering

1) Frequently accessed system components (Login, Browse)2) High volume of concurrent users (Add-to-bag, Checkout, Product Lookups)3) Resource intensive components (Create Order, Shipments, Waving of Orders)4) Business Critical Flows (Checkout, Pricing, Inventory)5) New hardware/software, system upgrades 6) Capacity Planning for peak season

What - Needs Performance Tested

1) Project Inceptiona. Understand project context and identify the risks

2) Iteration Planning a. Performance stories identified b. Prioritize features from the release plan

3) Story Timea. Ensure the story is playable and define the effortb. Clarify the need and timing for testing

4) Iteration Open/Close a. Open: teams commitment to the performance stories b. Close: review of the completed stories

5) Iteration a. Further elaboration b. Coding, Testing, Performance story signoff

6) Release a. Review of the performance results – GO/No GO!

How – To Performance Test

The Framework

1) Identify data injection points within the supply chain for problem isolationa. Queuesb. SOAPc. APId. Servletse. Database

2) Data Creation for performance, QA and development3) Faster Delivery of performance results 4) Measurable

a. Baseline vs. SLAb. Trendingc. Peak Volumes

5) ‘What If’ requests6) Simulators7) Timely data restore for further testing8) Capacity Planning 9) Reporting10) Performance test 24/7 with on shore and off shore

In SummaryThe Centre for Economic and Business Research (CEBR), which found that nearly £100 million of British e-commerce revenue is lost due to website failures and outages every year. The report shows that the UK’s e-commerce industry is worth £320 billion per year and growing at an unprecedented rate of 20 per cent year-on-year. The market is likely to grow even faster in the next ten years, as the proliferation of smart phone usage fuels more online transactions, meaning they stand to lose significant revenue and market share to rival firms if their websites fail. More than ever, your Website’s performance matters. The average online shopper expects your pages to load in two seconds or less, down from four seconds in 2006; after three seconds, up to 40% will abandon your site. According to studies by the Aberdeen Research Group, the industry average is 97.8% availability. Not bad right? Wrong. Consider what a two percent lack of availability really looks like: it means your site is out of business 8 days a year. For an ecommerce site generating $100,000 a day, that translates into a loss of $800,000 in yearly revenue. It’s not good enough for your site just to be available — it also has to be fast. The same Aberdeen study found that the average impact of a 1-second delay meant a 7% reduction in conversions. For the $100,000/day ecommerce site, a one-second delay means $2.5 million in lost revenues in a year.

1) When the ‘What If’ questions that arise at the end of a release can be answered – What if I increase the number of users or the number of transactions? What if I change the heap size? – how will the system perform?

2) When I can tune the key system areas on demand3) When you can performance test 24/7 – usually involving offshore4) With engineering activities and the processes of management formally defined, documented and

integrated into a process for continuous improvement5) When you have predictability modeling as part of the continuous improvement6) You have data injection points throughout the supply chain to run performance, QA or development

testing in the respective environments on demand7) When you find that your performance team is becoming a Share Service entity. You have been using the

companies data and now you have other teams coming to you to leverage the data. For example, training would like to use your data to run simulations for training purposes

8) When development begins incorporating performance into their development efforts; code reviews, profiling, writing to log files with key performance information

9) When you are able to have proper monitoring in place to provide key metrics for measuring 10) When you can test the network, http server, application server, database server as well as the application

code for performance11) When performance is leveraging database flashback technologies

How do I know when I have a Level 5 CMMI performance team?