Testing strategy for agile projects updated

QA & Quality Planningfor Agile projects

Agenda

• About QA/Testing• Some terminology• Traditional testing practices• Formulating a test strategy for the project• Iteration metrics• Success factors for quality in agile projects

What is Quality

"Quality is a customer determination. It is based on the customer's actual experience with the

product or service, measured against his or her requirements”

Software Quality Assurance(SQA)• Is a set of processes and methods to that is applied throughout the

software process to ensure quality of products– Comply with standards (ex: ISO 9001,CMMI)– Encompasses software development process

• Documentation• Reviews , Inspections• Testing• Configuration management, source control• Measurement and reporting

Software testing(quality control)

• Is a process of verifying and validating that it meets the business and technical requirements of the product being developed

•Also identifies important defects, flaws, or bugs in the application code that must be fixed.

Essentially we check for • Completeness• Fitness for purpose• Risks

Testing types– Unit testing

• Automatic test on code level run every night or after new code is added – to verify no bugs introduced

– Integration testing• Testing interactions between different modules of the system

– Feature testing• Performed by a QA when a feature is finished implemented

– System testing• To verify whether the system as a whole meets the specified requirements

– ExploratoryTesting• Exploratory testing is simultaneous learning, test design, and test execution.

– Regression Testing• Test to detect side effects from changes to the system, i.e after bug fix

– User acceptance testing• End user testing

– Compatibility testing• Verification of compatibility between AMX applications and the different OS, office applications, browsers and

databases etc that we support

How Traditional Test Practices Evolved

How Traditional Test Practices Evolved

Agile testing defined

• Agile testing is a software testing practice that follows the principles of Agile software development (remember agile manifesto..)

• Agile development integrates testing into the development process, rather than keeping it as a separate and distinct SDLC

• Agile testing involves a cross-functional Agile team

• Agile teams use a “whole-team” approach to “bake in quality” to the software product.

Role of the tester• Have an agile mindset• be an advocate to the end user– Understand the need of end user due to absence of

detailed specs– Participate in scrum meetings– Absorb user stories/ ask questions to clarify

• Working directly with devs, designers etc.• Cross functional• Expect changes and adapt

Test Strategy in agile • Have a test plan then get it validated and approved by whole team/ client

– Scope, resourcing, risks, test types, infrastructure, test environment test data, results

• Finding the right mix with testing methods. Have a right balance of • Automation

– CI– Unit– UI/API

• User acceptance testing(beta testing)• Exploratory testing

– Systems with heavy business logic favor test automation– Consumer-facing systems favor beta testing

• Building a foundation for core agile practices• Choose right people

Sample test plan

Test plan

Agile testing quadrant

Testing in agile • We start our testing activities at Release planning stage

• Release planning – Create test plans

• High level test planning • budgeting enough time

– Participate in story sizing“No story is done until its tested”

– Prioritizing “What stories finish by release”

Testing in agile • Each Sprint

– Participate in sprint planning– Estimate tasks(QA input)– Write test cases using stories(QA)

• High level test cases before coding begins- guide dev• Detailed test cases written during coding• Condition of satisfaction in user story

– Unit/ Integration tests(Dev)• Automatic test on code level run every night or after new code is added. Bugs found are fixed immediately

– Pair tests(Dev + QA)• Performed when a part of a feature is finished implemented. Bugs found are fixed immediately

– Feature tests(QA)• Performed when a feature is finished implemented. Errors found are logged in main project in bug system and

prioritized during next sprint planning. After all prioritized bugs are fixed, the feature is ready for System Test at the end .

– Reviews (stories, req, test cases with customer, programmer)• Increase collaboration and communications

Test cases

Test cases deriving techniques• Boundary testing• Equivalence classes• Decision tables• State transitional diagrams• Risk Analysis

Testing in agile • Each Sprint contd…

– Test automation• Automate new functional test cases• Run automated regression tests

– Perform non functional testing (load, security, usability etc.)• Non-functional tests can be included as US/tasks in the sprints

– Demo to the stakeholders

Agile test automation pyramid

Testing in agile • Hardening sprint (optional)

– A hardening sprint can be used for bug fixes in previous sprints. Bugs that are prioritized will be considered here

– Do it near the release

– The end game(System test)

Every member in the team is involved in this.

• Mock deploy on staging• Smoke test on staging• Perform non functional testing (load, security, usability etc)• Complete regression test• Perform UAT tests • Other tests

– Compatibility(DB upgrade etc)– Installation– Localization etc

Testing in agile • Release

– Define Exit criteria• Testing activities (bug severity, open bugs, coverage levels, metrics etc)• Product Artifacts (user guide, installation guides, other necessary

documentation are in place)

– Participate in retrospectives• Start• Stop• Continue

Testing-outlined

• Test strategy in agile development

Release planning

(test plan)

Each Sprint(Sprint QA activities)

Hardening Sprint

(system test)

Release QA activities

MetricsTo enhance predictability…– Measuring progress

• Burn down charts• Estimated vs. Actual time for tasks

– Defect metrics• Root cause analysis• Results from defect tracking system

– Traceability metrics• Test coverage (story vs. test cases)• Code coverage

Success factors for high quality agile projects

• Whole team approach• Automation (regression)• Collaboration with customer/stakeholders• Adopt agile mindset• Provide and obtain feedback• Building a foundation of core agile practices

Q&A

Thank you