View
215
Download
3
Category
Tags:
Preview:
Citation preview
Performance testing for Automation QA – why and how?
Kovalenko Andrey
Andrey Kovalenko6 last years in IT - again 3 years of developing and implementing distributed protected networks3 last years - performance engineer kovalenko.andrey.odessa
About
What is performance?
Load testing - to verify application behavior under normal and peak load conditions.
Stress testing - To determine or validate an application’s behavior when it is pushed beyond normal or peak load conditions.
Capacity testing - To determine how many users and/or transactions a given system will support and still meet performance goals.
Why do we need performance?
• Based non-functional aspects of the software system• Non functional testing should be performed after
functional testing• Almost impossible to run manually• Describes HOW GOOD the product works
Performance VS Functional testing
• Based on functional specification• Functional testing is executed first• Easy to do manual testing• Describes WHAT the product does
Functional manualFunctional manual
PerformancePerformance
Functional AutomationFunctional Automation• Based on functional specification• Automation should be performed after manual
functional testing• Impossible to run manually• Describes WHAT the product does
Software Testing Life Cycle
STLC for performance testing
1. Identify requirements1. Features related to performance – with devs2. Modules related to performance – with devs and FA/SA3. Analyzing application’s tiers
2. Acceptance criteria (Performance KPIs):1. Main TRT values – from your FA and public sources2. Main system resources threshold – from your experience
3. Strategy planning and design:1. Customer scenario use – from your FA2.2. Transaction defining Transaction defining – from your experience and FA3. Scripts writing4.4. Production-like environment establishing Production-like environment establishing – can be a problemcan be a problem5.5. Monitors for resources – native, from specific toolsMonitors for resources – native, from specific tools
STLC for performance testing
4. Execute & Analyze1. Run performance scenario2. Collect results and resources’ consumption
data3. Analyze results, comparing with KPIs4.4. Analyze source code to find possible Analyze source code to find possible
bottlenecksbottlenecks5.5. Analyze logs, heap GC, dumps Analyze logs, heap GC, dumps 6.6. Create recommendationsCreate recommendations for application for application
tuning – for application server, code, tuning – for application server, code, hardware, softwarehardware, software
7. Prepare reports for stakeholder
• Multi-user scenario• Interested in response time• Should be ran on production-like env• Resources must be monitored,
application should be profiled
Performance VS Automation
• Single user• Interested in response data• Can be ran on any hardware• No needs to monitor resources
AutomationAutomation PerformancePerformance
Automation->Performance(pretty straight ahead)
1. Exclude from you automation scripts data validation2. Allocate transactions you will measure in your code3. Frame these pieces of code in time measurement
constructions4. Get environment (real or cloud)5. Prepare set of monitors for system and application
resources6.6. Get a headache with running n scripts in parallel Get a headache with running n scripts in parallel 7.7. After running collect results and prepare report for After running collect results and prepare report for
stakeholderstakeholder
Automation->Performance(tricky way)
1. If you use NUnit, Junit, Selenium or HP UFT2. If you use Eclipse or VisualStudio3. HP LoadRunner has plug-in with ability to transfer your
automation scripts to performance – in this case you will be able to run performance scenario and create reports automatically
4. Get environment (real or cloud)5. Run scenario and get reports automatically
**LoadRunner has community license for 50 users for freeLoadRunner has community license for 50 users for free
Automation->Performance(tricky way)
Automation->Performance(tricky way)
Thank You!
Recommended