Cigniti joint webinar with Soasta - Agile DevOps: Test-driven IT Environment Management Solution

Are you Set for Black Friday?

Rajesh Sarangpani Asst. Vice President ,Cigniti Technologies Cigniti

Dan Boutin Senior Product Evangelist, SOASTA

June, 2015 2 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.

Agenda

Housekeeping & Introductions

Best Practices & Lessons Learned with SOASTA CloudTest

Customer Case Study with Cigniti

Questions

Wrap-Up


Retail E-Commerce Requirements

E-Commerce Testing Strategy in place Understand how the site/application is used

Rich Content (Rich Relevance) Third-party content Promotions! Social Media Coupons CDN

Require a good understanding of their users & customers


Retail E-Commerce Best Practices

Test in Production

Know Your Users

5 March 3, 2015 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.

SOASTA Global Test Cloud Worldwide Locations, SOASTA Centers of Excellence (COE)

17 Providers

54 Locations

800K Servers

500M Virtual Consumers * Additional Locations on demand


Automated Grid Provisioning

Your environment must be flexible & scalable


You need a Kill Switch – No Fear Factor!


Eliminate the Fear Factor

“We don’t test in production.”


You need drill down by Load Generation Location


Detailed Error Analysis

You need detailed error analysis - LIVE


Multi-Test-Run Comparison

Compare results of a LIVE test with previous test executions

You need to know: Are we better than last time?


Detailed Transaction, Page, and URL Analysis

• Detailed Transaction and Page Analysis of Web and Mobile Load Tests

• Detailed URL Analysis of Web and Mobile Load Tests

You need web & mobile analysis


Detailed Page Component Breakdown

Your analytics should have visibility into your CDN


Reflects growth in cloud hours – Amazon only! (17 other providers!)

7 Year Growth of cloud testing: SOASTA

….goodbye Fear Factor…

CONFIDENTIAL – Not for Distribution © 2015 SOASTA. All rights reserved. January 13, 2015

Fix Your process! No more outdated test creation

CONFIDENTIAL – Not for Distribution © 2015 SOASTA. All rights reserved. January 13, 2015

Analyze the most common session paths of real users


Possess a deep Real User Analytics capability Then take your mobile data analysis even further & deeper

Performance Cluster Analysis Performance Conversion Path Analysis

Dimension Co-Occurrence Analysis Conversion Rate by Load time for View Groups Analysis

18

Why is all this important?

• Revenue

• Brand

• Competitive advantage


Cigniti helping its retail clients achieve business objectives


Cigniti improves customer experience of one of America’s largest apparel retailer

Client is one of America’s largest Apparel and Home Furnishing retailers with over 1000+ stores across all the states and has a huge online presence with the ability to ship goods across the world

• Current Business model challenges (trying to evaluate its core)

• Attracting new customers v win over existing customers to buy more

• Significant erosion of online revenues

• Transformation from single channel to multi channel

• Coping up with outdated technology

• Poor Operational efficiencies


Client’s Testing Challenges

Business – IT collaboration Lock-in

Lack of visibility and transparency across the teams

Lack of comprehensive strategy to address holiday period peak demands

Availability and Reliability of current infrastructure was way behind industry peers

Testing as an afterthought

Specifically non functional testing was planned very late in the life cycle

Old testing methods, Insufficient test coverage and non availability of context specific tools

Testing across Channels, devices, understanding real user experience were some of the key gaps

Struggling to stand up test bed due to high capex involved

Testing was also conducted under lab conditions


Solution Summary – How did we address the challenges!

Moved from reactive to proactive approach– Implemented proven best practices

Realistic test coverage -Ensured 100% coverage by implementing log analytics

Improved operational efficiencies by moving away from on-premise to cloud based performance test beds and tools

Implemented focused testing cycles that carried out performance testing on production

Continuous Improvement : Proactive performance monitoring and feedback control

Created a framework to involve business teams throughout the lifecycle from planning to reporting


Plan: Visual Workload Model

Main business flows

% User mix across the flows

Average session time

Average/Peak concurrent

sessions/ hour

# of transactions/min (e.g.

Orders, bookings etc)

Geo distribution

Channel distribution

CDN versus non CDN traffic


Analyse & Design: Business KPI

Business KPI

Number of Orders

Per minute

Gross Margin

Total orders

per day,

week , month

New visitors

New customers

Abandonment

Metrics

(Cart, Checkout,

Home page)

Average

order value


Execute & Report: Actionable & Intuitive Business Reports

16.89%

25.03%

21.41%

13.97%

6.88%

4.28% 2.84% 2.26% 1.73% 1.29% 1.26%

0.61% 0.56% 0.24% 0.22% 0.12% 0.07% 0.02% 0.05% 0.05% 0.07% 0.05% 0.05% 0.05%

Time in Seconds

How long most of your users wait for Home Page?

When 1000 users are on your application, only 170 users are going to like it. 250 users might want to search for other options. 214 users are not going to come back. 366 users don't even see what is in your website.


IT reports: Detailed Drill down to fix the root cause


Improved Availability and Reliability of the architecture

5 X Improvement in Scalability

Reduced the response time across geo by 50%

Proactive performance monitoring solutions to aid timely action

Outcomes


So what value we can add ?

Smart Tools™ Test

Accelerators

Tool Partnerships

and Alliances Innovation

and IP

Enterprise

Frameworks Process Asset

Library

Test

Automation

Compatibili

ty Testing Globalization

Testing

Security

Testing

Regression

Testing

Performance

Testing

Functional

Testing

Test Advisory

Services

Test Center of

Excellence

Usability

Testing

Testing in

Agile

DevOps QA

Continuous

Testing Big Data &

Analytics

SOA & Web

Services

Core Services

Quality Engineering Services

Next Gen Services

Digital QA

Social Media

Testing

Mobile

Testing

Cloud

Migration

Enablers


CIO

SRA

AAT

TO

Independent Testing

IP Led Testing

Test Cloud Test Accelerators

Certified for Services

Agility

» Niche player

» Unbiased

» Industry Leader IV & V

» Certified & Passionate Career Testers

» IP-led testing services

» Deployed at no charge

» Accelerates events across

Test Life Cycle

» Pro Agile

» First organization to offer testing

services on cloud

» Flexible pay as you go model

» No upfront CAPEX

» Test accelerators at no charge

» ERP test accelerator for SAP and Oracle

» Test accelerator to support web 2.0 and

POS

» Reusable process asset

library

» World’s first CMMI-SVC v 1.3

» ISO 27001 Compliant

Delivery Centre

» 100% focus and specialization in

testing

» Flexible engagement models

» Agile in functioning

» Executive team involvement

Collaborate

Measure

Govern

Leverage

Innovate

Ensure

Global

Engageme

nt Model

What makes us different ?


Software

Cigniti joint webinar with Soasta - Agile DevOps: Test-driven IT Environment Management Solution