Testing in an Agile Context 2011

Testing in an Agile Context

“What he needs is some way to pay back. Not some way to borrow more.” - Will Rogers

Chris SterlingVP of EngineeringAgileEVM Inc.

Web: www.AgileEVM.comEmail: [email protected]

Blog: www.GettingAgile.com Follow Me on Twitter: @csterwa

Hash Tag for Presentation: #swdebt

Wednesday, June 8, 2011

http://www.SterlingBarton.com

http://www.SterlingBarton.com

mailto:[email protected]


http://www.GettingAgile.com

http://www.GettingAgile.com

© 2009-2010,

Chris Sterling – Sterling Barton, LLC

Partner, Sterling Barton, LLC

Author of Book “Managing Software Debt: Building for Inevitable Change”

Consults on software technology, Agile technical practices, Scrum, and effective management techniques

Certified Scrum Trainer

Innovation Games® Trained Facilitator

Open Source Developer2

Email: [email protected] Web: http://www.agileevm.comBlog: http://www.gettingagile.comFollow me on Twitter: @csterwa




http://www.agileevm.com


http://www.gettingagile.com




© 2009-2010,

Presentation Agenda

Agile - Find Issues Earlier

Effects of Quality Debt

Definition of Done

Quality Dashboard

Agile Test and Integration Strategies

Acceptance Test-Driven Development

Test-Driven Design

Agile Regression Test Triangle

Configuration Management Debt Strategy

Questions and Answers

3


© 2009-2010,

Why Use Agile Methods?

4


© 2009-2010,

“Stuff” Rolls Down Hill...

5

Maintenance

Integration

Testing(Validation)

Construction

Design

RequirementsSpecification


© 2009-2010,

“Stuff” Rolls Down Hill...

5

Maintenance

Integration

Testing(Validation)

Construction

Design

RequirementsSpecification

This is too late to find issues

and respond effectively


© 2009-2010, 6

The Agile Manifesto*

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on the right,we value the items on the left more.”

55

Kent BeckMike BeedleArie van BennekumAlistair CockburnWard CunninghamMartin Fowler

James GrenningJim HighsmithAndrew HuntRon JeffriesJon KernBrian Marick

Robert C. MartinSteve MellorKen SchwaberJeff SutherlandDave Thomas

* www.agilemanifesto.org


http://www.agilemanifesto.org/

http://www.agilemanifesto.org/

© 2009-2010, 7

Principles Behind the Agile Manifesto*1. Our highest priority is to satisfy the customer through early and continuous delivery of

valuable software.

2. Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.

3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

4. Business people and developers must work together daily throughout the project.

5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

7. Working software is the primary measure of progress.

8. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

9. Continuous attention to technical excellence and good design enhances agility.

10. Simplicity--the art of maximizing the amount of work not done--is essential.

11. The best architectures, requirements, and designs emerge from self-organizing teams.

12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

* www.agilemanifesto.org/principles 56


http://www.agilemanifesto.org/principles

http://www.agilemanifesto.org/principles

© 2009-2010, 8

An Agile Method:Scrum


Effects of Quality Debt

“Promises make debt, and debt makes promises.” - Dutch Proverb

9


© 2009-2010,

Effect of Project Constraints on Quality

10


© 2009-2010,

Effect of Project Constraints on Quality

10


© 2009-2010,

“For every [dollar] of competitive advantage gained by cutting quality, it costs $4 to restore it; and software is an organizational asset and decisions to cut quality must be made by executive management and reflected in the financial statements.” - Ken Schwaber

11


© 2009-2010,

Lack of emphasis on software quality attributes contributes to decay

12


© 2009-2010,

Software Quality Attributes Defined

13


© 2009-2010,

Software Quality Attributes Rating Tool

14


© 2009-2010,

Sonar: Quality Dashboard

15


© 2009-2010,

Agile Test and Integration Strategies

16


© 2009-2010, 17

Definition of Done

Defines work products that will be delivered with each feature as it is ready for acceptance.

Typical entries...

Code includes unit tests, reviewed, checked in

Tests described and executed

Build, release notes

Compliance documentation updated to include current functionality

What else?Example of “average” DoD


© 2009-2010, 18

Definition of Done as a Compliance Checklist

Acceptance defined criteria for each user story

Unit tests written and passed

Code compiles with no errors and no warnings

New code doesn’t break existing code

Test case review (Dev to review test case written)

Architectural impact assessed and artifacts updated if necessary

Comments in code

Error codes added

Code reviewed by peer

Code checked in with reference to US#/Task#

Tested on FE

Integration test written & passes

Test code reviewed

Environment requirements documented

Interface document updated/added and checked in to SVN

Acceptance criteria verified complete

All P1-P3 bugs for the story are closed

Test approves user story

Story demonstrated to product owner


© 2009-2010,

Another Definition of Done Example

19

Sprin

tSt

ory

Rele

ase

Unit TestsPassed

FunctionalTests Passed

AcceptanceTests Passed

Story andTask StatusUpdated

Build SystemCompiles

Code isReviewed

Code MeetsStandards

CodeImplements

Logging

CodeCommentsUpdated

Published toDev Server

ProductOwnerDemo

PastAcceptanceTests Passed

ProductOwner

Acceptance

No CompileWarnings in

Code

DemoPrepared

BugsCommitted in

Sprint Resolved

DeploymentDocs

Updated

ReleaseNotes

Updated

CodeRepositoryis Tagged

Release toStageServer

DeploymentTestingPassed

DeploymentDocs

Delivered

InfrastructureChange Notes

Delivered

IntegratedStress Testing

Passed

BuildRequirements

Met

ReleaseNotes

Delivered


© 2009-2010,

No matter what, the cost of addressing software debt increases with time.

20


© 2009-2010,

Acceptance Test-Driven Development

21


© 2009-2010,

Test-Driven Design (TDD) - Basic “Flow”

22


© 2009-2010,


22

Write Failing Test


© 2009-2010,


22

Write Failing Test

Make Test Pass


© 2009-2010,


22

Write Failing Test

Make Test Pass

Refactor to Acceptable Design


© 2009-2010,


22

Write Failing Test

Make Test Pass

Refactor to Acceptable Design


Test-Driven Design (TDD)

23

Lets go through an example session using TDD to drive the implementation of a user story to meeting its acceptance criteria.


© 2009-2010,

Jitter – Example TDD Session

Fake micro-blogging tool named “Jitter” is made by Seattle-based fictitious company that focuses on enabling coffee injected folks to write short messages and have common online messaging shorthand expanded for easy reading. The user story we are working on is:

So it is easier to read their kid’s messages, Mothers want to automatically expand common shorthand notation

The acceptance criteria for this user story are:

LOL, AFAIK, and TTYL are expandable

Expand lower and upper case versions of shorthand

24


© 2009-2010,

Expand LOL to “laughing out loud”

public class WhenMotherWantsToExpandMessagesThatContainShorthandTest {

@Test

public void shouldExpandLOLToLaughingOutLoud() {

JitterSession session = mock(JitterSession.class);

when(session.getNextMessage()).thenReturn("Expand LOL please");

MessageExpander expander = new MessageExpander(session);

assertThat(expander.getNextMessage(),

equalTo("Expand laughing out loud please"));

}

}

25


© 2009-2010,

But wait…what if…?

What if LOL is written in lower case?

What if it is written as “Lol”? Should it be expanded?

What if some variation of LOL is inside a word?

What if characters surrounding LOL are symbols, not letters?

Write these down as upcoming programmer tests as comments so I don’t forget them.

// shouldExpandLOLIfLowerCase

// shouldNotExpandLOLIfMixedCase

// shouldNotExpandLOLIfInsideWord

// shouldExpandIfSurroundingCharactersAreNotLetters

26


© 2009-2010,

Expand LOL If Lower Case

@Test

public void shouldExpandLOLIfLowerCase() {

when(session.getNextMessage()).thenReturn("Expand lol please");



equalTo("Expand laughing out loud please"));

}

This forced use of java.util.regex.Pattern to handle case insensitivity.

public String getNextMessage() {

String msg = session.getNextMessage();

return Pattern.compile("LOL”, Pattern.CASE_INSENSITIVE)

.matcher(msg).replaceAll("laughing out loud");

}

27


© 2009-2010,

Don’t Expand “Lol” – Mixed-Case

@Test

public void shouldNotExpandLOLIfMixedCase() {

String msg = "Do not expand Lol please";

when(session.getNextMessage()).thenReturn(msg);


assertThat(expander.getNextMessage(), equalTo(msg));

}

This forced me to stop using Pattern.CASE_INSENSITIVE flag in pattern compilation. Only use “LOL” or “lol” for now.

public String getNextMessage() {

String msg = session.getNextMessage();

return Pattern.compile("LOL|lol").matcher(msg)

.replaceAll("laughing out loud");

}

28


© 2009-2010,

Don’t Expand “LOL” If Inside Word

@Test

public void shouldNotExpandLOLIfInsideWord() {

String msg = "Do not expand PLOL or LOLP or PLOLP please";

when(session.getNextMessage()).thenReturn(msg);


assertThat(expander.getNextMessage(), equalTo(msg));

}

The pattern matching is now modified to use spaces around each variation of valid LOL shorthand.

return Pattern.compile("\\sLOL\\s|\\slol\\s").matcher(msg)


29


© 2009-2010,

Expand “LOL” If Not Inside Word

@Test

public void shouldExpandIfSurroundingCharactersAreNotLetters() {

when(session.getNextMessage()).thenReturn("Expand .lol! please");



equalTo("Expand .laughing out loud! please"));

}

Final implementation of pattern matching code:

return Pattern.compile("\\bLOL\\b|\\blol\\b").matcher(msg)


30


© 2009-2010,

The Agile Regression Testing Triangle*

31* The Agile Triangle has been modified from Mike Cohn’s original version


© 2009-2010,



Automated Unit TestsMake up largest portion ofregression tests and are

developed by programmers


© 2009-2010,





Integration TestsAutomated &Exploratory


© 2009-2010,





Integration TestsAutomated &Exploratory

Smoke++ TestsRisk-based UI &API Automated

Tests


Configuration Management Debt

“If releases are like giving birth, then you must be doing something wrong.” - Robert Benefield

32


© 2009-2010,

The Power of 2 Scripts: Deploy and Rollback

33


Thank you

Questions and Answers

34


© 2009-2010,

Chris Sterling – Sterling Barton, LLC

Partner, Sterling Barton, LLC

Author of Book “Managing Software Debt: Building for Inevitable Change”

Consults on software technology, Agile technical practices, Scrum, and effective management techniques

Certified Scrum Trainer

Innovation Games® Trained Facilitator

Open Source Developer35

Email: [email protected] Web: http://www.agileevm.comBlog: http://www.gettingagile.comFollow me on Twitter: @csterwa










Technology

Testing in an Agile Context 2011