Stéphane Landelle & Pierre Dal-Pra

eBusiness Information,Groupe Excilys

Performance & Load Testing

Part 1Concepts

Define performance?

• fast?• robust?• resource effective?

=> Define your requirements!

Web speed is a user experience

• < 0,5 s : loading looks instantaneous

• 0,5-2 s : not instantaneous, but acceptable

• > 3 s : users start leaving your site

Put things into perspective

Product Owner requirement :

Every page should show up under 500ms

WRONG !

User expectations = per use case!

Speed matters

• More traffic

• Better conversion rate

• Less angry users

Some figures

Google :Results/page : 10 => 30

-> +500 ms

-> -20% pages seen

Amazon :+100 ms

-> -1 % sales

estimated annual cost : $160M

Load tests : why ?

Load tests

=

Anticipate production problems

+

Train yourself on your app & infrastructure

Methodology

Clicking everywhere/Selenium = not a load test !

• You expect more than 1 user

• "Seems fast enough." => Not a metric !

Methodology

Comes after proper local perf testing

• Client/Network optimization: minification, javascript, sprites, caching...o Tools : Chrome Developer Tools, YSlow, Google

PageSpeed Insights...

• Application debugging: 50 SQL queries/page = BUG!o Tools : VisualVM, Yourkit, JProfiler, MAT...

Methodology

• Analyze = Data = Monitoring

• Fix hotspot

=> Iterate!

Tooling

• Load injector: Gatling, JMeter, Locust.io...

• JVM monitoring: JMXtrans, Yammer Metrics...

• System monitoring: Nagios

• Network monitoring

• Database monitoring

• Dashboard: Graphite, Ganglia

• Webapp mock: H. Gomez's basic perf webapp

Types of load tests

• Capacity Test

• Stress Test

• Endurance Test

Capacity Test

Goal : Determine how much load your system can hold

How :Repeat the same scenario over and over, but add virtual

users every time until performance starts degrading.

A response time under 1 s is expected

-> user cap : 1500 users

Stress Test

Goal :Study system behavior in case of heavy load, during AND

after

How : Find the max load your system can handle, and then run

the scenario with a heavier load.

10k users for 1 min, then 1k : webapp struggles, but stabilizes

Endurance Test

Goal :Validate system behavior after a long period of activity.

How :Run the scenario with a manageable load, but for a long

period of time (several hours at least).

Fast memory leak :Runs fine for 2 minutes until heap's full...

Ramps

Start virtual users progressively, because that's what real users do !

Ramps also help to warm up your system.

Reports

The purpose of load injectors : stress your app and produces reports

• Meaningful reports help developers analyze stress tests results and what to make of them

• Something shiny to give to your boss

The good, the bad and the ugly metrics

Every metric can be useful, but some less than others...

• Response time min/max = worst case/best case

• Mean can be biased in case of extreme values

Response time is a physical phenomenon

=> Statistically distributed

Percentiles to the rescue

nth percentiles

=

n % of users' response time

Part of the development process

Like functional testing, load testing should be :

• integrated early in the development cycle

• automated

• versioned

Part 2Gatling

Yet Another Stress Tool

JMeter, Grinder, Tsung, LoadUI,

LoadRunner, Neoload…

High PerformanceIssue #1

http://www.shopfbparts.com/catalog/nal-19201331_w.jpg

1 user = 1 thread

With 50 threads on a JVM

With 2000 threads on a JVM

Blocking I/O

Threads? Waiting…

… and sleeping

Is that a real problem?

Can you trust your results?

JMeter 2.8 perf test, expecting 300 tr/sec

UsabilityIssue #2

Listen, it's not that complicated...

Graphical User Interface

MaintainabilityIssue #3

What was this change about?

Gatling can change all that!

Version 1.4.3Released February 2013

Say hello to my little friend…

Be asynchronous, embrace the actor model

Use non-blocking I/O

• Async HTTP Client• Netty

Scenario = Code (Scala) = DSL

Easy

Use the rich DSL…

Checks

• regex / css / xpath / jsonPath

• find / findAll / count

• is / in / not / whatever

Structures

• doIf / repeat / during / asLongAs

• randomSwitch / roundRobinSwitch

Error handling

• tryMax / exitBlockOnFail

Feeders

• csv / tsv / jdbc

… or write your own Scala code…

…or use the Recorder

Integrations• Maven Plugin• Maven archetype (run in IDE)

• Jenkins plugin

• Graphite live reporting

Coming soon…

• Websockets, JDBC…• Clustering

Part 3Demo

Conclusion

Really efficient?

Jmeter perf test run with Gatling, expecting 300 tr/sec

http://gatling-tool.org

https://github.com/excilys/gatling

@GatlingTool

https://github.com/slandelle

@slandelle

https://github.com/pdalpra

@pierre_dalpra