Performance Engineering Next Level

By Deepti Bhutani

Enable Performance Testing through the entire development cycle to ensure adequate performance of the code by the end of project

Define the acceptance criteria and definition of done or pass/fail of a story. Build the performance testing into smaller cycles and call them unit performance

testing that will measure component level performance . Build performance test cases in the tools like Jira or any other tool to enable visibility. Ensure that the Agile team are confirming the release against the agreed set of

performance requirements. Ensure continuous feedback among all stakeholders .

Performance Testing Approach in Agile

Groups Tasks Define how work will be broken down into small

chunks. Performance Team lead identifies performance

testing stories and make sure the acceptance criteria is defined.

Resource Planning for sprint work. Need to train the performance testers to adapt to

agile approach.

Tasks Create performance test cases. Create scripts run tests measure performance and

report performance Create front end performance measurement using

web page test or equivalent .

Agile Performance Team Structure

Scrum Level

Program

Level

Sprint Flow

Performance Activities Create the performance test cases in shared repository for each sprint in parallel with

coding. Work on unit level performance test of each module being developed . Report the individual module performance to scrum team Automate the tests explore the tools that will work better to include as part of CI Work on getting the test data created for regression test

Environments Can be performed in QA environment after the code has been QA certified

Tools We can explore the tools like JunitPerf . We can also include the load testing scripts as part of the CI build.

Resources Agile revolves around principle of cross training and working more in collabaration so this can

be achieved by developers also to begin with once the performance engineer get ramped up.

Sprint Level Performance Testing

Performance Activities Create the performance test cases in shared repository for the hardening sprint. Work on performance testing integrated components like services along with the UI Report the performance of the integrated components. Automate the tests and run the integrated tests(Single users tests) as part of CI To

track performance. Continue to work on getting the test data created for regression test We may have this sprint or might create integration user stories as part of next sprint.

Environments Can be performed in QA env after the code has been QA certified

Tools Use load runner or Jeter be flexible with tool choices

Resources Performance test engineer

Performance Testing In Hardening Sprint

Performance Activities

Fully integrate yourself into the Sprint team, immediately   Performance awareness, practices, and results organically increased because it was a well represented function within the team as opposed to an after thought farmed out to a remote organization.

Build multiple performance and stress test scenarios with distinct goals and execution schedules.  Plan for longer soak and stress tests as part of the release process, but have one or more per-sprint, and even nightly, performance tests that can be continually executed to proactively measure performance, and identify defects as they are introduced.  Consider it your mission to quantify the performance impact of a code change.

Extend your Continuous Integration (CI) pipelines to include performance testing .  At PayPal, I built custom integrations between Jenkins and JMeter to automate test execution and report generation.  Our pipelines triggered automated nightly regressions on development branches and within a well-understood platform where QA and development could parameterize workload, kick-off a performance test and interpret a test report.  Unless you like working 18-hour days, I can’t overstate the importance of building integrations into tools that are already or easily adopted by the broader team.  If you’re using Jenkins, you might take a look at the Jenkins Performance Plugin.

Define Key Performance Indicators (KPIs).  In an Agile model you should expect smaller scoped tests, executed at a higher frequency.  It’s critical to have a set of KPIs the group understands, and buys into, so you can quickly look at a test and interpret if a) things look good, or b) something funky happened and additional investigation is needed. Some organizations have clearly defined non-functional criteria, or SLAs, and many don’t. Be Agile with your KPIs, and refine them over time. Here are some of the KPIs we commonly evaluated:

Environments Needs to be performed in pre prod after the code has been QA certified

Tools Use load runner

Resources: Performance Test engineer

Performance Testing In Regression Sprint

Change Agile from a development practice to a delivery practice Build a change-friendly framework Bake performance testing into every iteration Performance awareness, practices,

and results will organically increase if it is a well represented function within the team as opposed to an after thought

Automate as soon as possible Extend your Continuous Integration (CI) pipelines to include performance testing In an Agile model you should expect smaller scoped tests, executed at a higher

frequency.  It’s critical to have a set of KPIs the group understands, and buys into, so you can quickly look at a test and interpret if a) things look good, or b) something funky happened and additional investigation is needed

Monitor performance trends per iteration Monitor SLA performance per iteration

Summary