Agile testing principles and practices - Anil Karade

Agile Testing – Principles and Practices

Anil Karade

Talentica Software (I) Pvt. Ltd.

Agenda Overview of Traditional Processes Agile Methodology Scrum Agile Testing Principles Agile Testing Practices With Case Study Agile Management Tools Used Benefits

How Traditional Test Processes Work

Traditional Planning :

Design + DevAnalysis Testing

Analysis Design + Dev Testing

Release

Release

Actual :

Traditional Testing Practices

“Last Defender of Quality” Stance Detailed preparation and upfront planning Heavyweight documentation Strict entrance and exit criteria with signoffs Attempts at process enforcement

What is Agile ? Agile – A continuous stream of business values

Agile methods in contrast to traditional ones produce completely developed and tested features at frequent intervals of 2-4 weeks.

Iterative approaches mean we can trade features for time instead of sacrificing quality

S1 S2 S3 Sn

Release Release Release

Completed, potentially shippable FeaturesDesigned, Coded and Tested

Release

Waterfall Vs Agile

Feature Usage

Values Expressed in Agile Manifesto:

Individuals and interactions over processes and tools

Working software over comprehensive documentation.

Customer collaboration over contract negotiation Responding to change over following a plan.

Scrum Scrum in game of Rugby

Agile principles will not work without integrated team mindset

Scrum

Scrum Scrum - an Agile approach to software development Scrum projects make progress in a series of Sprints Sprint - Timeboxed Period (2 – 4 weeks) During Sprint, team does:

− Analysis, Design, Code and Test Product is potentially shippable after every Sprint

Agile Testing - Principles Shifting QA Focus From Being last line of defense to providing information,

feedback and visibility.

The adverse relationship between testers and developers must be replaced with a spirit of collaboration.

Principles Contd… Quality Responsibility –

• Traditional model – Responsibility of independent QA group

• Agile Model - Quality is everyone’s (team) responsibility”. QA execute test cases, developers do too.

Testing Phase-• Traditional model – Testing is a distinct phase that

happens in the end• Agile model - QA teams test continuously to measure the

quality and provide feedback.• Quality is not added later. It evolves gradually.

Principles Contd…

Shortening Feedback Loops

Agile Testing Practices with Case Study Scrum Implementation

• We follow 2 Weeks Sprint• Release sprint (1 week) before going to production• At the end of Release sprint final release to production is done

Agile Testing Practices with Case Study

Challenges Faced• Change in mindset• Fear to lose identity• Collaboration with programmers and customers• Not Understanding Agile Concepts – ‘Scrummerfall’

Scrummerfall

Testing Practices QA Role in Release Planning

In contrast to traditional processes testers are involved from day one of the project.

Sizing Stories – QA gives estimate for testing effort for the upcoming stories. In some cases testing a piece of functionality might take longer than coding it.

Prioritizing – Stories that have many unknown areas or have high business impact should be planned in an early iteration, so there is time to recover if story blows up.

Infrastructure – Plan for CI setup, test environment, test data set up

Testing Practices Contd...Clear Requirements = Half Done

Practice we followed -

Full Kit Teams -• Separate full kit team was formed that included BA, Dev, QA

experts and Customer. • This team worked on getting the requirements clarified from

everyone’s point of view. • This ensured that we are minimizing the risk of finding the

defects that will challenge the requirements in the end stages.

Testing Practices Contd...

QA Role at Iteration Kick -Off :• Learn stories in detailsExample Story

Story – 1As a customer I want to know how much my order will cost to Ship based on the shipping speed I select so that I can choose a different shipping speed if I want to.

Testing Practices Contd... QA Role at Iteration Kick -Off :

• Writing Task Cards –

• Review high level test cases with developers

Dev tasks for Story -1• Coding for Shipping API• UI Design• UI coding

Testing tasks for Story -1• Write high level test cases• Write detailed test cases• Acceptance testing for API• Detailed functional testing for API• Exploratory testing• Automation


Coding and Testing :• QA works closely with programmers, customers and other team

members to produce stories in small testing-coding-reviewing-testing increments.

• Write detailed tests for a story as soon as coding begins. Follow conversational test case creation approach for writing test cases

Contd…

Testing Practices Contd... Conversational Test Case Creation –

• Create business facing test cases in conversation with BA. e.g. business flows overlooked by BA

• Create technology facing test cases in conversation with Developers. e.g. browser dependencies, performance, etc

• Creating test cases in isolation will not assure coverage.

Testing Practices Contd…

Identify Risks / Prioritize Test Cases • With Agile testing QA not only needs to test sooner, but they

need to uncover high risk areas faster. • Execute the test cases in order of the risk associated with it, so

that at the end of iteration we do not end up in finding critical bugs where we don't have sufficient time to fix.


# Item Impact Probability Risk

1 Incorrect cost displayed 4 2 8

2 User can’t choose different shipping option

5 1 5

3 Estimated cost doesn’t match actual cost at checkout

3 4 12

4 Invalid postal code entered and not caught by validation

4 1 4

5 User can’t change shipping address

5 2 10

6 User changes shipping address, but cost doesn’t change accordingly

5 4 20

Risk Analysis for Story -1


Test Early (Behind GUI) –• Always ask programmers how a small set of coding effort can be

tested earlier. One of the examples we followed is API testing.• With traditional approach QA used to test only in stable builds,

whereas with agile we are uncovering the risks earlier by API testing, database testing


Behind GUI (API) Testing –

Testing for Shipping cost API

Assumption – only valid destinations are considered as input

Start with simple happy path testing-

Weight Destination Postal Code

Cost ($)

5 lbs 80104 7.25

Testing Practices Contd… Coding and Testing Progress Together –

Iterative back and forth in coding and testing.

Try Canadian postal code. Result – It returns exception

Revised happy path test –

Any Clarification - Programmer + QA + Customer = Power of Three

Weight Destination Postal code

Country Code Cost

5 lbs 80104 US 7.25

5 lbs T2J 2M7 CA 9.40


• Tests that critique the product – Be aware of all nice to have, enhancement requests.

Don’t jeopardize other stories by spending too much of time adding things that doesn’t have a big ROI.

Pair Testing

Which bugs to Logs ?


Exploratory Testing• Exploratory testing is done when we get a working model of a

functionality• Simultaneously develop and execute test cases while exploring

functionality

Testing Practices Contd... Lightweight Documentation

Instead of writing verbose, comprehensive test documentation focus on essence of test.

Use collaborative tools – wiki, Review board etc.. Centralize documents – e.g. centralize generic test cases Leverage documents for multiple purpose


Automation Automation is the key to agile. Automation helps QA in focusing on other important

aspects of testing like functional, exploratory testing. Focus on automating business logic bypassing GUI when

UI is changing continuously. Have customizable smoke tests Use lightweight automation tools for automating unit tests

Tools used: JUnit, Selenium

Testing Practices Contd... Release Sprint Testing

• Following testing activities need to occur during release sprint before finally shipping the product.

• Final pass of end-to-end system integration test• Final performance/security test• Documentation touch ups• User Acceptance Test

• Time required is greatly reduced (from initial 3 weeks to 1 week, in our case ) as compared to traditional process and less likelihood of any surprises.

Testing Life Cycle Sprint 1 (2 Weeks)

Test Analysis, Acceptance Test,

Investigative Test, Automated Regression Test

Release Sprint (1 week)end-to-end system Test

Final Performance /Security TestUAT

Production

Sprint 2 (2 Weeks)Test Analysis,

Acceptance Test, Investigative Test,

Automated Regression Test

Investigative Test : - Exploratory Test - Scenario Test - System Test

Agile Tools

We used VersionOne and JIRA for scrum management.

Excel backlog sheets were also used. http://agilesoftwaredevelopment.com/scrum/simple-product-backlog

Agile Implementation Benefits

Increased customer satisfaction On time deliveries without compromising quality Iteration wise testing makes QA more focused on

specific functionality which results in effective testing.

Due to stable builds test cases execution was fast Reduction in number of defects observed in testing

Thank You

Technology

Agile testing principles and practices - Anil Karade