Performance Testing With Loadrunner

Paul Pu

What on this course

Performance Planning/Testing Load Test Planning Load Test Reporting Load Testing Products LoadRunner Architecture LoadRunner Script Framework LoadRunner Scripting Script Generation From Spreadsheet Performance Monitors LoadRunner Reports LoadRunner Results Explorer LoadRunner Error Listings Performance Tuning Virtual Table Server Load Balancing Servers Mercury Capacity Planning Mercury Diagnostics

How to teach you

20 % theory, 80 % practice Project Oriented

Real Life project Not a demo Not other person’s Website You can access source codes, table data

Real working environment

Who is going to teach you

Paul Pu likes to share his experience with you

20 years working experience in Software Development Master in Computer Science, Master in Information System, Master

in Management and Planning Part time Professor in State University of New York 1998-2000 Part Time Professor in Humber College 2001 – 2005 IBM Certified Instructor Selected Publications: An Internet Based Medical Archive System,

IEEE; Medical Data on Demand with WebMIA, IEEE

What kind of projects are we going to test

CRM application: Contact Management System http://www.objectsolution.com:7777/cmdb_test/

Job Search Engine

http://www.objectsolution.com:7777/jobmining_test/

System Management System http://www.objectsolution.com:7777/tctweb_test/

What kind of reference I can get

You will get work reference from Object Solution if your testing work is satisfied by them

What kind of background I need

Some programming background

Database Concept and Basic SQL

Functional Testers

Functional Testers

Provide a spreadsheet listing each screen of the application. Identify what aspects of the application does not work. Identify need for volume testing.

Alert if transaction times are noticibly less or more than anticipated.

Performance Engineer

Performance Engineer

Evaluates application risks and determines measurement and test approach, in accordance to pre-defined strategies and project plans.

Creates multi-user load scripts, conducts runs, generates analysis (provides expertise on LoadRunner)

Identifies areas of concern. Provides technical status on testing activities. Provides developers expertise on the use of performance-impacting

techniques.

Types of Performance Testing/Engineering

Speed Tests (for Responsiveness) ContentionTests (for Robustness) Volume Tests (for Extendability) Stress / Overload Tests (for Sustainability) Fail-Over Tests (for Resilience & Recoverability) Spike "Peak-Rest" or "Daily" Tests Endurance "Soak""Longevity" Tests (for Reliability) Scalability (Efficiency) orReconfiguration Tests Availability (Schedulability)

Speed Tests (for Responsiveness)

During speed testing, the user response time (latency) of each user action is measured.

The script for each action will look for some text on each resulting page to confirm that the intended result appears as designed.

Since speed testing is usually the first performance test to be performed, issues from installation and configuration are identified during this step.

Because this form of performance testing is performed for a single user (under no other load), this form of testing exposes issues with the adequacy of CPU, disk I/O access and data transfer speeds, and database access optimizations.

Speed Tests (for Responsiveness)

Accomplishments Identified the business processes under test.

Documented each user action to be measured.

Documented production installation configuration instructions and settings.

Quantified the start-up, shut-down, and user GUI transaction response (latency) times when the system is servicing only a single user at a time (under no other load) in order to determine whether they are acceptable.

Ensured CPU, disk access, data transfer speeds, and database access optimizations are adequate.

Contention Tests (for Robustness)

This form of performance test aims to find performance bottlenecks (such as lock-outs, memory leaks, and thrashing) caused by a small number of Vusers contending for the same resources.

Each run identifies the minimum, average, median, and maximum times for each action. This is done to make sure that data and processing of multiple users are appropriately segregated.

Such tests identify the largest burst (spike) of transactions and requests that the application can handle without failing. Such loads are more like the arrival rate to web servers than constant loads.

Contention Tests (for Robustness)

Accomplishments

Identified performance bottlenecks (such as lock-outs, memory leaks, and thrashing) caused by a small number of Vusers contending for the same resources.

Ensured that data and processing of multiple users are appropriately segregated.

Identified the largest burst (spike) of transactions and requests that the application can handle without failing. Such loads are more like the arrival rate to web servers than constant loads.

Volume Tests (for Extendability)

This form of performance testing makes sure that the system can handle the maximum size of data values expected.

These test runs measure the pattern of response time as more data is added.

These tests make sure there is enough disk space and provisions

for handling that much data, such as backup and restore.

Volume Tests (for Extendability)

Accomplishments

Quantified the degradation in response time and resource consumption at various levels of simultaneous users. This is done by gradually ramping-up the number of Vusers until the system "chokes" at a breakpoint (when the number of connections flatten out, response time degrades or times out, and errors appear).

Determined how well the number of users anticipated can be supported by the hardware budgeted for the application.

Quantified the "Job flow balance" achieved when application servers can complete transactions at the same rate new requests arrive.

Ensured that there is enough transient memory space and memory management techniques.

Make sure that admission control techniques limiting incoming work perform as intended. This may include extent of response to Denial of Service (DoA) attacks.

Stress / Overload Tests (for Sustainability)

This form of performance testing determines how well the number of users anticipated can be supported by the hardware budgeted for the application.

This is done by gradually ramping-up the number of Vusers until the system "chokes" at a breakpoint (when the number of connections flatten out, response time degrades or times out, and errors appear).

During tests, the resources used by each server are measured to make sure there is enough transient memory space and adequate memory management techniques.

This effort makes sure that admission control techniques limiting incoming work perform as intended. This includes detection of and response to Denial of Service (DoA) attacks.

Stress / Overload Tests (for Sustainability)

Accomplishments

Quantified the degradation in response time and resource consumption at various levels of simultaneous users.

Determined how well the number of users anticipated can be supported by the hardware budgeted for the application.

Quantified the "Job flow balance" achieved when application servers can complete transactions at the same rate new requests arrive.

Ensured that there is enough transient memory space and memory management techniques.

Make sure that admission control techniques limiting incoming work perform as intended. This may include extent of response to Denial of Service (DoA) attacks.

Fail-Over Tests (for Resilience & Recoverability)

This form of performance testing determines how well (how quickly) the application recovers from overload conditions.

For example, this form of performance testing ensures that when

one computer of a cluster fails or is taken offline, other machines in the cluster are able to quickly and reliably take over the work being performed by the downed machine.

This means this form of performance testing requires multiple identical servers to be configured and using Virtual IP addresses accessed through a load balancer device.

Fail-Over Tests (for Resilience & Recoverability)

Accomplishments

Determined whether the application can recover after overload failure.

Measured the time the application needs to recover after overload failure.

Spike "Peak-Rest" or "Daily" Tests

This form of performance testing determines how well (how quickly) the application recovers from overload conditions.

For example, this form of performance testing ensures that when one computer of a cluster fails or is taken offline, other machines in the cluster are able to quickly and reliably take over the work being performed by the downed machine.

This means this form of performance testing requires multiple identical servers to be configured and using Virtual IP addresses accessed through a load balancer device.

Spike "Peak-Rest" or "Daily" Tests

Accomplishments

Determined — through suddenly adding and then completing transactions — that the app releases memory.

Endurance "Soak" "Longevity" Tests (for Reliability)

This form of performance testing makes sure that the system can sustain -- over at least a 24 hour period -- a consistent number of concurrent Vusers executing transactions using near peak capacity.

Because longer tests usually involve use of more disk space, these test runs also measure the pattern of build-up in "cruft" (obsolete logs, intermediate data structures, and statistical data that need to be periodically pruned).

Longer runs allow for the detection and measurement of the impact of occasional events (such as Java Full GC and log truncations) and anomalies that occur infrequently.

These tests verifies provisions for managing space, such as log truncation "cron" jobs that normally sleeps, but awake at predetermined intervals (such as in the middle of the night).

Endurance "Soak" "Longevity" Tests (for Reliability)

Accomplishments

Ensured that the system can sustain over at least a 24 hour period a consistent number of concurrent Vusers executing transactions using near peak capacity.

Measured the pattern of build-up in cruft (logs, data structures, and statistics that need to be periodically pruned).

Detected the impact of occasional events (such as automatic cache flushes, Java Full GC and log truncations) and anomalies that occur infrequently.

Make sure there is enough disk space and provisions for managing space, such as log truncation jobs that only occur automatically in the middle of the night.

Scalability (Efficiency) or Reconfiguration Tests

This form of performance testing involves repeating tests above on different server/network hardware configurations to determine the most cost-effective option to support targeted load levels (one aspect of Capacity Planning).

The outcome of scalability efforts feeds a spreadsheet to calculate how many servers the application will need based on assumptions about demand.

Scalability (Efficiency) or Reconfiguration Tests

Accomplishments

Determined — through repeated tests on different server/network hardware configurations — the most cost-effective option to support targeted load levels (one aspect of Capacity Planning).

Availability (Schedulability)

This form of performance testing provides a continuous assessment of the availability and speed of key user actions.

These are run on applications in production mode.

This provides alerts when thresholds are reached and trends to guage the average and variability of response times.

Quick Demo

Gen