Upload
alon-girmonsky
View
324
Download
8
Embed Size (px)
DESCRIPTION
Performance Testing in the Agile Age An organization That relies on web or mobile applications to deliver services, benefits and features needs to intro agility into its product development lifecycle. In order to Achieve Agility we have to build applications today differently than we did it in the past. Bugs, bad user experience and downtime are luxuries That are far more expensive today than in the past. cloud-based performance testing Accommodates gift requirements to validate application performance, surface performance related bugs, bottleneck and bad user experience. It Allows for a swift resolution and testing over and over again without having to wait.
Citation preview
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
Performance Testing in the
Agile Age
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
Alon Girmonsky About The Speaker
• Hands-On Developer for the Past 20 Years • Expertise
– Large-Scale Web and Mobile Applications – IP & Internet
• Founder & CEO of BlazeMeter – A Load Testing Platform as a Service – 100% Compatible with Apache JMeter
USERS
20,000
TESTS
500,000
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
The Agile Enterprise
• All organizational divisions - not just R&D • It’s about survival in a highly competitive
business environment • Innovate in short Iterations – a MUST! • New DNA • New IT Infrastructure (a.k.a Devop)
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
Agility Enablers
Open Source
Cloud Compu/ng
E.G E.G
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
Performance Testing
Waterfall Agile
Front-‐end
Back-‐end
Func/onal
Load E.G API
E.G Java Script
In The Agile Age
2005 2010 2014
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
Agile? What it Means?
"We'd like to squeeze in a load test for XXXX’s biggest event of the year, Fantasy Football season. Our best opportunity to execute this is tonight 3-7am EST. We'd like to generate a constant 40,000 hits/sec for about 1 hour. .."
Traditionally this would have taken weeks and cost $$$
An Example from a Large Media Company
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
Agile? What it Means?
Traditionally this would have taken weeks and cost $$$
Network Bottleneck Identified
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
For Both Mobile and Web
Massive Global Simulation of Users, Devices and Mobile Networks
Agile Performance Testing
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
480 CPU Cores and
1.8 TB of Memory
What Beast Has .. ?
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
480 CPU cores and 1.8 TB of Memory A Typical Test
EC2 Large 2 cores, 7.5GB
EC2 Large 2 cores, 7.5GB
EC2 Large 2 cores, 7.5GB
EC2 Large 2 cores, 7.5GB
EC2 Large 2 cores, 7.5GB
EC2 XL 4 cores, 15GB
X 200
X 20
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
Mobile Application Simulation A Common Use-Case
Using Agile Performance Testing
• You can create such test in a matter of minutes • You can run dozens of times per day on demand • You can enjoy an interactive reporting that provides an
immediate intelligence
Actions Sign-up Sign-in Purchase Browse
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
Comprehensive scripting at your fingertips
Any Script, Any Scenario
• HTTP / HTTPS • Login / Signup • Form POSTing / Credit card transactions • Web services • Parameters extraction • Mobile • Mongo, MSSQL, Oracle SQL, MySQL • Multiple user profiles
» Any script » Any scenario » Any protocol
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
Agile Summary
1 to 300,000 Concurrency
Open-Source Compatible
Self-Served
TTS < 10m
Any Protocol
Any Scenario
Multi Locations
On-Demand
Comprehensive
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
The Load Sensitivity Point What to look for?
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
Load Performance Types
Perfect Time
Sensitive
Bad
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
Load Performance Types
Perfect
The Perfect Load
Time
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
Load Performance Types
Sensitive
Sensitive to Load
Time
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
Load Performance Types
Bad
Bad Performance
Time
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
Load Performance Types
Perfect
Here They Are Again
Time
Sensitive
Bad
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected]
What Can Be Achieved By Using Agile Performance
Testing
Apache JMeter + AWS or Commercial Vendors
Use Case Examples
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
What to Look Out For? GEO Sensitive Testing
• Realistic Vs Unrealistic Simulation • Single Location Vs Geo Distributed Load • Small Subset of Target Servers • Unrealistic Connectivity
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
Testing AKAMAI CDN GEO Sensitive Testing
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
Testing AKAMAI CDN (ALL) GEO Sensitive Testing
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
Testing AKAMAI CDN (VA Only) GEO Sensitive Testing
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
In Real-time Pinpoint Accuracy
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
In Real Time Pinpoint Accuracy
50%-50%
60%-40%
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
In Real Time Pinpoint Accuracy
100 hits/s
1,000 hits/s
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
Benefits Distributed Load
• More realistic • More load • Evenly distributed • More IPs (For a more even load balancing) • More locations (For GEO sensitive systems)
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
Conclusions Agile Performance Testing
• Cloud Computing + Open Source = Agile • Any Protocol, Any Scenario, Any Scale • Self-Served, On-Demand, TTS < 10m • For Dev, Devop, Performance Engineer and Ops • Test Driven Development • Better Quality + Faster = Competitive !
175 Varick Street NY, NY 10013|1.855.445.2285|[email protected] @alongir
Thank you for your time!
Jolt Awards 2013: The Best Testing Tools