Session 

 

Presented by: 

Jim Hirschauer 

  

Brought to you by: 

  

340 Corporate Way, Suite   Orange Park, FL 32073 888‐2

W14 Concurrent4/9/2014   3:15 PM     

“Techniques for Agile Performance Testing”  

 

AppDynamics   

    

300,68‐8770 ∙ 904‐278‐0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com 

Jim Hirschauer AppDynamics  

Technology evangelist for AppDynamics Jim Hirschauer has an extensive background in highly available, business critical, large enterprise IT operations environments. Jim has been interested in application performance testing and monitoring since his days as a systems administrator in a retail bank. His passion for performance analysis led him down a path to design, implement, and manage the cloud computing monitoring architecture for a top-ten investment bank. During his tenure at the bank, Jim created new processes and procedures that reduced the time spent in the test and QA cycle while simultaneously increasing overall code release quality and dramatically improving end user experience.

2/18/2014

1

PERFORMANCE TESTINGCRASH COURSE

Jim Hirschauer@HirschOnAPM

Lockheed Martin Aeronautical Systems

Duke Energy

Wachovia

Wells Fargo

AppDynamics

2/18/2014

2

Wh dWhy does performance

matter?matter?

Mi ft f d th t BiMicrosoft found that Bingsearches that were 2 seconds

slower resulted in a 4.3% drop in revenue per userdrop in revenue per user

2/18/2014

3

Wh M ill h d 2 2When Mozilla shaved 2.2seconds off their landing page, Firefox downloads

increased 15.4%increased 15.4%

Making Barack Obama’s website 60%

faster increased donation conversions

by 14%

2/18/2014

4

Amazon and WalmartAmazon and Walmartincreased revenue 1% for

every 100ms of improvementimprovement

2/18/2014

5

Performance directly impacts the bottom

linee

2/18/2014

6

T l f th t dTools of the trade for performance

testingtesting

2/18/2014

7

Understand your baseline performance

Static

vs

Hello World

vsvs

App

2/18/2014

8

Apache Bench

ab -c 10 -t 10 -k http://dustinwhittle.com/

2/18/2014

9

Server Software: nginxServer Hostname: dustinwhittle.comServer Port: 80

Document Path: /Document Length: 6642 bytes

Concurrency Level: 10Time taken for tests: 10.042 secondsC l 286Complete requests: 286Failed requests: 0Write errors: 0Keep-Alive requests: 0Total transferred: 2080364 bytesHTML transferred: 1899612 bytesRequests per second: 28.48 [#/sec] (mean)Time per request: 351.133 [ms] (mean)Time per request: 35.113 [ms] (mean, across all concurrent requests)Transfer rate: 202.30 [Kbytes/sec] received

Connection Times (ms)Connection Times (ms)min mean[+/-sd] median max

Connect: 9 20 14.2 17 148Processing: 117 325 47.8 323 574Waiting: 112 317 46.3 314 561Total: 140 346 46.0 341 589

Percentage of the requests served within a certain time (ms)50% 34166% 35675% 36680% 372

Siege

2/18/2014

10

siege -c 10 -b -t 10S http://dustinwhittle.com/

** SIEGE 2.72** Preparing 10 concurrent users for battle.The server is now under siege...Lifting the server siege... done.

Transactions: 263 hitsAvailability: 100.00 %Elapsed time: 9.36 secsData transferred: 0.35 MBResponse time: 0.35 secsTransaction rate: 28.10 trans/secThroughput: 0.04 MB/secConcurrency: 9.82ySuccessful transactions: 263Failed transactions: 0Longest transaction: 0.54Shortest transaction: 0.19

2/18/2014

11

Crawl the entire app to discover all urls

sproxy -o ./urls.txt

2/18/2014

12

SPROXY v1.02 listening on port 9001...appending HTTP requests to: ./urls.txt...default connection timeout: 120 seconds

wget -r -o verbose txt -l 0 -t 1 --spider -w 1 -ewget -r -o verbose.txt -l 0 -t 1 --spider -w 1 -e robots=on

-e "http_proxy = http://127.0.0.1:9001""http://dustinwhittle.com/"

sort -u -o urls.txt urls.txt

2/18/2014

13

Benchmark traffic across all unique

urls with siegeu s t s ege

siege -v -c 50 -i -t 3M -f urls.txt -d 10

2/18/2014

14

Apache JMeter

2/18/2014

15

M lti M h i iMulti-Mechanize is an open source framework for performance and load

testingtesting

2/18/2014

16

pip install multi-mechanize

multimech-newproject demo

2/18/2014

17

import requestsimport requests

class Transaction(object):def run(self):

r = requests.get('http://dustinwhittle.com/')d()r.raw.read()

class Transaction(object):def run(self):

br = mechanize.Browser()br.set_handle_robots(False)

start_timer = time.time()resp = br.open('http://www.dustinwhittle.com/')resp.read()latency = time.time() - start_timerself custom timers['homepage'] = latencyself.custom_timers[ homepage ] latency

start_timer = time.time()resp = br.open('http://www.dustinwhittle.com/blog')resp.read()latency = time time() start timer

2/18/2014

18

[global]run_time = 10

5rampup = 5results_ts_interval = 1progress_bar = onconsole_logging = offxml_report = on

[user_group-1]threads = 1script = demo.py

multimech-run demo

2/18/2014

19

2/18/2014

20

What about when you need more than

one machine?o e ac e

Who lives in the cloud?

2/18/2014

21

Bees with Machine Guns

A utility for arming (creating)A utility for arming (creating) many bees (micro EC2

instances)to attack (load test)

t t ( b li ti )targets (web applications)

2/18/2014

22

pip install beeswithmachineguns

# ~/.boto

[Credentials]

aws_access_key_id=xxxaws_secret_access_key=xxx

[Boto]

ec2_region_name = us-west-2ec2_region_endpoint = ec2.us-west-2.amazonaws.com

2/18/2014

23

bees up -s 2 -g default -z us-west-2b -i ami-bc05898c -k appdynamics-dustinwhittle-aws-us-west-2 -l ec2-

user

Connecting to the hiveConnecting to the hive.Attempting to call up 2 bees.Waiting for bees to load their machine guns.......Bee i-3828400c is ready for the attack.Bee i-3928400d is ready for the attack.The swarm has assembled 2 bees.

2/18/2014

24

bees report

Read 2 bees from the roster.Bee i-3828400c: running @ 54.212.22.176Bee i-3928400d: running @ 50.112.6.191

2/18/2014

25

bees attack -n 1000 -c 50 -u http://dustinwhittle.com/

Read 2 bees from the roster.Connecting to the hive.Assembling bees.Each of 2 bees will fire 50000 rounds, 125 at a time.Stinging URL so it will be cached for the attack.Organizing the swarm.Bee 0 is joining the swarm.Bee 1 is joining the swarm.Bee 0 is firing his machine gun. Bang bang!Bee 1 is firing his machine gun. Bang bang!Bee 1 is out of ammo.Bee 0 is out of ammo.Offensive complete.

Complete requests: 100000Requests per second: 1067.110000 [#/sec] (mean)Time per request: 278.348000 [ms] (mean)50% response time: 47.500000 [ms] (mean)90% response time: 114.000000 [ms] (mean)

Mission Assessment: Target crushed bee offensive.The swarm is awaiting new orders.

2/18/2014

26

bees down

What about the client side?

2/18/2014

27

Google PageSpeed

Google PageSpeed Insights

2/18/2014

28

2/18/2014

29

Google PageSpeed API

curl "https://www.googleapis.com/pagespeedonline/v1/runPagespeed?url=http://

dustinwhittle.com/&key=xxx"

2/18/2014

30

WBench

gem install wbench

2/18/2014

31

wbench http://dustinwhittle.com/

2/18/2014

32

Automate client-side performance testing

with Gruntt G u t

Use Bower (forUse Bower (for dependencies),

Grunt (for automation),and Yeoman (for bootstrapping)

2/18/2014

33

2/18/2014

34

H lHow many people understand exactly how

fast their site runs in production?production?

2/18/2014

35

2/18/2014

36

2/18/2014

37

2/18/2014

38

2/18/2014

39

http://webpagetest.org/

2/18/2014

40

2/18/2014

41

L d t tiLoad testing services from the

cloudcloud

2/18/2014

42

2/18/2014

43

I t t t t dIntegrate automated performance testing into

continuous integration for server-side and client-sideserver side and client side

2/18/2014

44

U d t d thUnderstand the performance implications of

every deployment and package upgradepackage upgrade

Monitor end user experience from end to end in productionto e d p oduct o

2/18/2014

45

Questions?