Slide 1

Innovations and Adaptations in Agile Testing

Subrahmaniam S R V ACP, PMP CSM

25th February, 2014

http://knowledgehut.com/blog/wp-content/uploads/2013/04/agilepicture.gif

Slide 2

What will we cover today?

Agile Testing – Role of testersr

Key Agile Testing practices

Outcome assessment

Lispin‘s Agile testing quadrants & challenges

Adaptations to handle these

Disclaimer: The views expressed are that of the presenter and do not necessarily represent the views of the company he works for.

Slide 3

Agile Testing - Role of testers

Testers‘ objective is to enable timely release of quality software. They find defects and verify the fixes

as part of this objective

Test early

Sensitise developers

towards planned tests

Focus on buggy or critical areas

Automate alongside during

Sprints

Reduce # of open defects

Collaborate with developers

(reproducing defects, bug

verification etc.)

Slide 4

Testing upfront moves the powers the project ahead

Slide 5

Continuous Integration and Automated test

Image: http://www.agilenutshell.com/assets/continuous-integration/continuous-integration.png

Nightly builds followed by automated tests Code to verification cycle will be less than a day. Automated suite should grow with every Sprint. Focus on Critical functional areas for automated System test

Slide 6

Test throughout the Sprint duration, not just at the end

Image: Agile Testing by Elisabeth Hendrickson

The user stories are to be tested throughout the Sprint, not just when they are completed (mini-waterfall) – and prevent clogging at the end.

Slide 7

Testing is everyone‘s responsibility!

Developer tests Unit tests

QA practices (Pair Programming, static code analysis, CI etc.)

Automated tests

Tester‘s tests

Need for dedicated testers is progressively reduced in mature agile teams

Slide 8

Co-location of test team members

Developers, testers and product management teams sit in the same area.

More face-to-face communication and less reliance on emails.

Quick resolution of queries and open issues

http://www.environmentalevaluators.net/2011/07/key-concepts-of-complexity-in-science-feedback-loops/

Slide 9

Crispin’s Agile Testing Quadrants – Challenge in executing these within Sprints

Especially in cases with High domain / technical complexityhttp://img3.douban.com/view/page_note/large/public/p14837373-1.jpg

Slide 10

Typically, the right quadrant tests (that which critique the product) are done only in dedicated test phases due to logistic issues

Sprinting phase Dedicated test phases

ReleaseCode Freeze

Full System tests Full Load and

Performance tests Other non-functional - ility

tests Exploratory tests Compliance to regulatory

standards and guidelines

Critical defects found during dedicated test phases can put the release in quandry, because they can be hard to fix and found too late!

Slide 11

Challenges emerging from the right quadrant tests! Can these be scheduled during Sprints?

http://img3.douban.com/view/page_note/large/public/p14837373-1.jpg

Slide 12

Nature of exploratory and non-functional tests

Highly specialized set of tests

Defects hard to uncover and hard

reproduce

Environment and tool set-up likely to be

complex

Defects may have high turn-around time

Defects can potentially affect release

schedules

May require architecture fixes

May need long duration test cycles

Slide 13

Considerations in scheduling Exploratory and Non Functional tests

Accomodating these factors in the Agile test framework vis-a-vis what tests get carried out within Sprints is the critical decision

Sprint Test

Limited feasibility

to do within Sprint Software

may not be ready for full-blown

tests

Critical tests -

need to be performed at earliest

Slide 14

One possible recommendation – Test in Parallel, with a Sprint lag!

Sprint 1 Sprint 2

Functional tested package

Sprint 3 Dedicated test phases

System & NFR test

The possibility of any critical defects being found in dedicated test phases is reduced

Defects

System & NFR test

.....

Product

Backlog

Hardening Sprints

Or

Release Sprints

Slide 15

Why this approach?

A stable build and undisturbed time window is provided for right quadrant tests done along-side Sprints

1

Any bug found early will go towards avoiding late surprises in dedicated test phases

2

Teams for Sprint tests and right quadrant tests can be groomed for their skills accordingly

3

Duration of the dedicated test phases is optimized, by doing critical tests early.

4

Intent is to ensure certainty of the release date. 5

Testers doing the right quadrant tests often need specialized skills – be it in Load and Performance, domain or in NFRs

6

Slide 16

Evolving Definition of Done

Complex

Moderate

Simple

Functional tested

System testedProduction ready

Syst

em C

ompl

exity

Done Criteria

1

2

3

The gap between Sprint Done and being Production ready will be tested during the hardening sprints.

It is the same as the gap between Potentially shippable and shippable

1

2

3

Web application for internal use

Device centric multi-environment thick client

Domain centric external facing web application

Slide 17

Manage Test environments

Have parallel or shadow (to system test) test environments for testing during Sprints

These will also be of use for developers to test their fixes when long test phases are ongoing during dedicated test phases

Primary ST environment Shadow environments for use by Scrum Team

Slide 18

Re-skilling and competency building

- Automation has to go hand in hand during the Sprints.

- Automation engineers need to think beyond standard tools and look for custom built test suites working in tandem with standard tools

Building self-organized teams

Test Automation Tooling Problem solving Inter-personal skills

- Identify areas where tools can be applied

- Evaluation of new tools for data capture and analysis

- Project planning and scheduling

- Problem identification and evaluation of alternatives

- Work scoping, constraint and risk management

- Communication and Presentation skills

- Team work- Negotiation skills

Training areas

Building technical Competence

Slide 19

Items for Optimization

Automation &

Continuous

Integration

Test

scheduling

Environment,

Tools and

Test stubs

Team skilling

Increased use of white box test

techniques and custom built test

stubs

Slide 20

Status & Progress Indicators

the Contra indicatorsthe positives

• Key end-to-end and Non Functional requirements are tested early

• Defects in Discussion / Clarification / Dispute mode are reduced

• Developers (along with testers) doing informal testing in the test environment

• Testers supply all the details need for developers for bug fixing: Logs, environment, data snapshot etc.,

+• Critical defects found late –

impacting release date• High number of open defects• Increase in escape defects -

Testers getting influenced by development team‘s thinking (?)

• Number of stories – Not done due to test being incomplete

-

S R V Subrahmaniam, SDLC professional and Agile Specialist

in.linkedin.com/in/subrahmaniamsrv/

@srv_subbu

Srv.subbu@yahoo.in