Gareth Lynn SOASTA - cloudbees.comLyn… · © 2014 SOASTA CONFIDENTIAL - All rights reserved. 0 Gareth Lynn SOASTA Webinar

0© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Gareth Lynn

SOASTA

Webinar


Rapid Delivery

Agile development

Continuous

delivery

DevOps principles

Mobile and Web

Online everything

Mobile commerce

Legacy modernization

Complexit

y

Composite apps

3rd party

services

API proliferation

Unprecedented

Scale

100’s to millions

Local to global

100%

availability


o Small Batch Sizes

o Comprehensive Version Control

o Automation

o Continuous Feedback

o Emphasis on working builds

o Consistent environments

o Developer/Tester/Ops Collaboration



50,000 Apps 1,000,000 Apps 1,500,000 Apps



“We didn’t have enough

testing for a very

complicated project”

US Health and Human Services

Secretary, Kathleen Sebelius


Mobile Test

Automation

Performanc

e Testing

Real Test

Devices

Solid

Integration


Does it work as

designed?

Does it perform

under extreme

circumstances?

Approve materials Real Drivers

What happens

in actual

conditions?

Assembly Line


Functional

validation

Performance,

reliability,

scalability tests

Real Users/Real Devices

Build

What happens

in actual

conditions?

Unit-tested code


o Rapid & accurate test development

o Reliable, maintainable test automation

o Stability across mobile OS platforms

o Device performance measurement

o Real-Time feedback via Continuous Integration


o Fully managed device cloud installed

behind the firewall

o Real-time device inventory, status and

availability

o Online access for all your teams

o 24x7, secure, remote access to devices

o Manual & Automated functional test

o Remotely access to selected mobile

device and any mobile app

• Use devices for manual testing and monitoring

• Remote screen-share devices for collaboration

on design or issues


o Dedicated Public Device Cloud

o Access to the latest market leading

devices (iOS/Android)

o Automatic installation, maintenance

and upgrade

o Hosted devices in a secure data

center

o Stock devices, not jail

broken/rooted

o Seamless integration with

TouchTest and CI environments

o API available to install apps,

reboot/wake-up devices, launch

testsPerfect for 24x7 Automated Mobile Regression Testing


s

Memory leaks

Test Scale

Sta

ge / T

eam

Dev

& Test

Release

& Deploy

Network

& OpsCDN file placement

Load Balancer configuration

Network bandwidth

Network configuration

DNS routing

Inadequate server resources

Default configuration settings

Unbalanced web serversAuto-scaling failures

Latency between systems

Slow third-party plug-ins

Garbage collection

Database thread counts

Inefficient database queriesSlow pages

Conflict with other apps

Test Lab Staging Production (100% +++)

Search technology limits

Method-level tuning

Max sockets exceeded

Firewall max capacity

Global latency variance

Security bottlenecks

Cloud Testing at All StagesPerformance validation from development to production

Continuous tuning for full-scale…and well beyond


Analytics

CloudTest

Databas

e

Main

Perf Tester 1

Analytics

Load

GeneratorLoad

GeneratorLoad

Generator

San Francisco (GoGrid)

Analytics

Load

GeneratorLoad

GeneratorLoad

Generator

Tokyo (AWS)

Analytics

Load

GeneratorLoad

Generator

Amsterdam (Azure)

Data sources:

Native (SSH), JMX, PerfMon, CA Introscope,

AppDynamics, New Relic, CloudWatch,….

Web

Server

Web

Server

App ServerWeb

Server

Web

Server

App Server

App Server

App Server

Load

Balancer

Cache

Cache

Databas

e

Analytics

Load

GeneratorLoad

Generator

Virginia (IBM)

Analytics

Load

GeneratorLoad

GeneratorLoad

Generator

SOASTA

Global

Test Cloud

Access

for all

Analytics

Load

GeneratorLoad

GeneratorLoad

Generator

East Coast (AWS)

“Conductor”

CDNCredit

Card

Web

analytic

s

Other

3rd pty

svcs

Chicago

(Rackspace)


Use real user data to

iterate dev & test plans

Run iterative load

tests via Jenkins

Daily (or more frequent) updates

keep teams quality focused

Embedded failure

details & bottleneck

data isolate issues

Manage to a

performance

baseline

Mobile? Run functional

automation on real devices

for functional & performance

validation

Spin up suitable, consistent

test environments in the

cloud or the lab

Jenkins runs any

unit tests as usual



Ut

Unit Testing

Pe

Performance

Testing

Fu

Functional

Testing

Development Testing Operations

TIME

Test

Dev Ops

Traditional Linear Approach

Continuous Testing Approach


SOASTA Customers

“4 months of manual tests now run in 1 hour, overnight.”

“25% reduction in time to market for production

rollouts.”

Steven Winter, Director of Quality

FIS Mobile – Mobile banking platform to over 1000 banks


o 250 hours of manual tests running in just over 1

hour every night

o Regression Tests not run until feature complete

now run nightly – 4 months to overnight

o Fixing bugs earlier – Detailed test results for

developers

o 25% Reduction time to market

o Reduced field failures


SOASTA Customers

“TouchTest has been critical in improving both the speed to deliver

and the quality of our mobile apps.”

“We decreased our test cycles from three weeks to a couple of days.”

Thierry Chassaing, SVP of Product & Engineering.

ItsOn - the leader in Mobile Smart Services.


o Understand your requirements

o Identify the cloud advantages for you

o Automate the obvious and most critical

o Connect to a continuous process

o Align teams with actionable information

























www.SOASTA.com

Documents

Gareth Lynn SOASTA - cloudbees.comLyn… · © 2014 SOASTA CONFIDENTIAL - All rights reserved. 0 Gareth Lynn SOASTA Webinar