Seven Keys to Navigating Your Agile Testing Transition

ial

Presented by:

Bob Galen Vel rs

Brought to you by:

340 Corporate Way, Suite Orange Park, FL 32073 888‐2

MD AM Tutor4/7/2014 8:30 AM

“Keys for Transitioning to Agile Testing”

ocity Partne

300,68‐8770 ∙ 904‐278‐0524 ∙ [email protected] ∙ www.sqe.com

http://starcanada.techwell.com/�

An agile methodologist, practitioner, and coach based in Cary, NC, Bob Galen helps

rship

.

Bob Galen ers Velocity Partn

guide companies in their adoption of Scrum and other agile methodologies and practices. Bob is a principal agile evangelist at Velocity Partners, a leading agilenearshore development partner; president of RGCG; and frequent speaker on software development, project management, software testing, and team leadeat conferences and professional groups. He is a Certified Scrum Coach, Certified Scrum Product Owner, and an active member of the Agile and Scrum Alliances. In2013 Bob published Scrum Product Ownership–Balancing Value from the Inside OutReach him at [email protected].

http://starcanada.techwell.com/sme-profiles/bob-galen�

mailto:[email protected]

1

Keys for Transitioning to Agile TestingTesting

Myths & Realities from the Trenches

Bob GalenPresident & Principal Consultant

RGCG, LLC [email protected]

IntroductionBob Galen

Independent Agile Coach (CSC) at RGCG, LLC

Principle Agile Evangelist at Velocity Partnersp g g y

Somewhere ‘north’ of 30 years overall experience ☺Wide variety of technical stacks and business domainsDeveloper first, then Project Management / Leadership, then TestingSenior/Executive software development leadership for 20 yearsPracticing formal agility since 2000XP, Lean, Scrum, and Kanban experienceFrom Cary, North CarolinaConnect w/ me via LinkedIn and Twitter @bobgalen

Copyright © 2014 RGCG, LLC 3

Connect w/ me via LinkedIn and Twitter @bobgalen

Bias Disclaimer:Agile is THE BEST Methodology

for Software Development…However, NOT a Silver Bullet!

2


Outline – Myths & Realities

Introduction1. Transforming your

Team9. Developer to Tester

WorkflowTeam2. Automation 3. Developers &

Automation4. Developers Testing5. Test Planning &

Scripts6. Testing within the

Workflow10. Managing Agile Testers11. Test Metrics12. Retrospectives – The

Secret Sauce13. Continuous Improvement14. The Customer15 Agile Requirements The


gSprint

7. Exploratory Testing8. Role of Testers

15. Agile Requirements – The Product Backlog

3-Pillars of Agile Testing & Quality

3

#1, Transforming your team

Myth: You need all programmers or highly technical testers when you move to agile

Reality: A mix is best –Manual, domain-centric and technical skillsS i / i ti kill


Some programming / scripting skillsSoft / collaborative skills

Reality: And throw out all of that Developer-to-Tester ratio ‘stuff’.

#2, Automation

Myth: You need 100% automation to startagile testingagile testing.

Reality: You simply need to have a strategy AND doggedly pursue automation where it makes sense

Make it part of the Backlog and work it every sprint

Reality: There are some excellent Open Source tools


that supplement agile automation developmentReality: The Agile Test Automation Pyramid is the right overall strategy

4

Agile Test Automation PyramidMike Cohn; Lisa Crispin & Janet Gregoryhttp://behaviordrivendevelopment.wikispaces.com/Testing


Brainstorm…Agile, Multi-tiered Automation

Get together in small groups of 4-6 to discuss

Take a few minutes and think about your current automation approaches:

Tooling, approaches & strategies, strengths, weaknesses, opportunities, maintenance challenges, future technology, etc.

What sorts of adjustments would you need to make to take this approach?take this approach?What would be the largest challenges in taking this approach? How might you overcome them?Do you “buy” the whole-team view to automation?


5

#3, Developers & Automation

Myth: QA designs, writes & runs all of the test automationautomation

Reality: Everyone should be responsible for automationDevelopers need to minimally attend to Unit LevelParticipate in any framework or re-use developmentWriting ‘glue’ code – fixtures, step files, etc.

Reality: It also extends into your Build & Continuous


Reality: It also extends into your Build & Continuous Integration systems

All automation should be ‘wired’ into CIDashboards, trending, lava lamps, etc. for all to see…

#4, Developers Testing

Myth: Developers can’t test their own code—they’re not independent enough nor skilled enough to do it properly.

Reality: We need to stop stereotyping team members, their strengths and their abilities.

Developers can absolutely test their own code.Some are better at it than othersPair with them to help test appropriately


6

#5, Test Planning & Scripts

Myth: You don’t need to plan (it just happens )(it just happens…)

and you don’t need functional test cases (automation takes care of everything…)

Reality: Plans help the team focus on the risk-based testing required within an iteration AND across a release


Reality: Scripts (test cases) help formalize and drive your testing;

Absolutely required in regulatory environments

Reality: You’ll never actually automate every test

Brainstorm…Agile Planning & Execution

Get together in small groups of 4-6

Take a few minutes discuss your current planning and test process mechanisms.

What would an Agile Test Plan “look like” in your organization?What would Test Cases “look like”? What about progress measures? And traceability?Can you move from the “individual” to the “team”?Can you move from the individual to the team ?

Be prepared to share…


7

#6, Testing within the Sprint

Myth: You simply need to run 100% of the tests within the constraints of the Sprint…that’s“Agile”

Reality: Rarely possible in most contexts. You first need a high-degree of automation and business support (for example: equipment costs)Very mature test automation and CI / CD environments


Reality: Most agile teams adopt some sort of risk-based testing approach for within the sprints

Dealing with Technical Test DebtThen leverage Hardening / Stabilization pre-release sprints

The Agile Release TrainSynchronized

Internal Release

External Release

Iterate

Iterate

Team 1

Team 2

Team 3

Iterate

Iterate

Harden Iterate Iterate Iterate

X-teamHarden

Harden

HardenIterate Iterate

Iterate Iterate

Iterate Iterate

Iterate

Iterate

Docs,Training,Support,

UAT,Comp.

Continuous Integration



Copyright © 2014 RGCG, LLC

Team 4 HardenIterate Iterate Iterate Iterate Iterate

Team n

…



16

8

The Agile Release TrainExample: eCommerce / SaaS Model

External Release

10 days 10 days 5 + 2 days

Iterate

Iterate

Team 1

Team 2

Team 3

Iterate

Iterate

Harden

X-teamHarden

Docs, Training

Harden

HardenIterate Iterate




Rinse

&

Repeat


Team 4 HardenIterate IterateTeam 8…Continuous Integration

Continuous IntegrationEnvironment

Evolution Dev + QA Dev + QA QA -> Staging Production

17

The Agile Release TrainExample: iContact / SaaS Model

Production Release

3 weeks / 15 days 4-5 days

SBET, Exploratory –Regression Testing

Team 1

Team 2

Team 3

Iterate

Iterate

Harden

X-teamHarden

Docs, Training

Harden

HardenIterate




Rinse

&

Repeat


Team 4 HardenIterateTeam 10…Continuous Integration

Continuous IntegrationEnvironment

Evolution Dev + QA QA -> Staging Production

9

Brainstorm…“Your” Agile Release Train


Take a few minutes and think about your current release constraints

timing, customers, domain, competition, # of teams, technology, etc.

Design a release train model for your organizationO l it ith t ti ti iti l d il tOverlay it with testing activities, plans, and milestonesPresent it to your larger table/group; gain feedback & adjustBe prepared to share…


#7, Exploratory Testing

Myth: There is no place for Session Based ExploratoryMyth: There is no place for Session Based Exploratory Testing in agile contexts.

Reality: ET and SBET are a beautiful complement to agile testing.

Helping nurture pairing & collaboration across teams and functionsfunctionsDefining new (more valuable) test casesQuickly gaining quality & usability feedback

Let’s explore the details of SBET…


10

#8, Role of Testers

Myth: That the testers alone own quality & testingMyth: That the testers alone own quality & testing practices within each team and sprint

Reality: The testers foster a “Whole Team” view towards quality—focusing less on “Testing” and more on “Quality Practices & the Customer”

S i id f th t T ti th “h d bit ”


Serving as guides for the team; Testing the “hard bits”Facilitating exploratory testing sessions—finding more interesting / valuable testsWorking with the Product Owners—are we solving the customers problems?

#9, Developer to Tester Workflow

Myth: There is always a hand off from developers toMyth: There is always a hand-off from developers to testers; usually quite late in the sprint. That’s simply the “way of things” in software development.

Reality: Scrummer-fall is alive and well…but, Wrong! Teams need to swarm on their work, as flow & throughput matter the most


throughput matter the most.WIP limits and close proximity / collaboration help establish a healthy tempo of developer & tester pairingMicro-handoffs – testing as development progresses!Do you log bugs? Or do you fix bugs?

11

#10, Managing Agile Testers

Myth: The functional test manager is in charge of deciding how who when etc for the testin charge of deciding how, who, when , etc. for the testteam.

Reality: You still absolutely need functional leadership within agile teams;

However, it’s focused towards quality practices, strategy & hi d h dli i di t / l ti


coaching, and handling impediments / escalationsEncouraging transparency, transforming metrics & reportingSupporting & protecting the teamsEncouraging risk-taking, innovation & creativity (Slack Time)

Levels of Done-Ness Criteria

Activity Criteria Example Basic Team

Work Products Done’ness criteria Pairing or pair inspections of code prior to check-in; or

development, execution and passing of unit tests.

User Story or Theme Level

Acceptance Tests

Development of FitNesse based acceptance tests with the customer AND their successful execution and passing. Developed toward individual stories and/or themes for sets of stories.

Sprint or Iteration Level

Done’ness criteria Defining a Sprint Goal that clarifies the feature development and all external dependencies associcated with a sprint.

Release Level

Release criteria

Defining a broad set of conditions (artifacts, testing activities or coverage levels, results/metrics, collaboration with other groups, meeting compliance levels, etc.) that IF MET would mean the release could occur.

24Copyright © 2014 RGCG, LLC

12

Brainstorm…“Your” Definition of Done


Using the 4-tier approach referenced start filling in the 4 levels as a group.Consider any criteria you are currently using at your companies?Also consider current issues or challenge you might have where “done-ness” would help?And what about Ready-ness?Be prepared to share…


#11, Test Metrics

Myth: You can and should move forward reporting everything exactly asforward reporting everything exactly as you have before.

Including any ‘dysfunctional’ metrics that your process and/or PMO dictates.

Reality: The metrics should change immediately. F QA d T t t i t d T C t i t i (V l


From QA and Test centric towards Team-Centric metrics (Value, Throughput, Quality, Team)Stop reporting out on “Testing”; it’s irrelevant!This effects planning as well—estimation, progress, risk, etc.Contribute quality-centric Information radiators to the mix

13

Brainstorm…Morphing your Metrics


What are you measuring today? Why?How are they driving your success and behaviors?

As you move to agile, what can/should you be measuring in the 4 key areas:

Value, Quality, Throughput & Predictability, and Team?


How will you change your existing metrics? What behaviors are you trying to inspire? Be prepared to share…

#12, Retrospectives: The Secret Sauce

Myth: Testers are “Second Class” citizens who don’tMyth: Testers are Second Class citizens who don t play an active part in the project & team

Reality: There are many places to “make a difference”Getting the 800 lb. Gorillas out on the table; Showing courage; telling truthFostering continuous improvement within the teamFostering continuous improvement within the teamSetting the example; showing vulnerability—admitting you’re wrongTeam listening; active planning; dependencies; pairingRisk-taking; Failure!


14

#13, Continuous Improvement

Myth: We’re generally ‘stuck’ in our approaches so just accept them and doapproaches so just accept them and do the “best you can”.

Reality: Continuous improvement is everyone’s responsibility—to engage, suggest, take ownership of current results, explore root causes, etc.

Active participation in your teams Retrospectives is a key way to guide quality, testing, and customer-centric improvements.Courage!


#14, The Customer

Myth: Business Analysts capture customer requirements and testers test them forcustomer requirements and testers test them forcompleteness.

Reality: You need to begin to partner with the Customer – Stakeholders – Product Owners to produce software that solves the their problems.

Move to the “front” and help define & refine User Stories with your Product OwnerActively participate in Sprint ReviewsShow value for automation; placing test investments in the Backlog


15

#15, Agile Requirements –The Product Backlog

Myth: We can’t start testing until the requirements areMyth: We can t start testing until the requirements are finished or stable; no matter how ‘agile’ we are.

Reality: Hogwash! Get over it…Ambiguity and incompleteness need to become your friend and ally. As does working with your Product Owners and Customers to


As does working with your Product Owners and Customers to help define the requirementsRealizing that the requirements (User Stories) are only complete at the end of each sprint.

Brainstorm…Agile Requirements


Are iterative, are intentionally incompleteThe “older” the are, the larger and less defined they areEnter the sprint at 70%, exit at 100%Drive questions, dialogue, discussion, and collaboration; think 3 Amigos or the Triad


So, WHY? And how will you make this work as a tester?

Be prepared to share…

16

Agile Test Transformation Strategy:3 Pillars of Agile Quality


3 Pillars of Agile Quality

Development & Test Automation

• Pyramid-based Strategy: (Unit + Cucumber +

Software Testing

• Risk-based testing: Functional & Non-Functional

Cross-Functional Team Practices

• Team-based Pairing

Selenium)

• Continuous Integration

• Attack technical infrastructure in the Backlog

• Visual Feedback –Dashboards

• Actively practice ATDD and BDD

• Test planning @ Release & Sprint levels

• Exploratory Testing

• Standards – checklists, templates, repositories

• Balance across manual, exploratory & automation

• Stop-the-Line Mindset

• Code Reviews & Standards

• Active Done-Ness

• Aggressive Refactoring of Technical Debt

• User Stories, “3 Amigo” based Conversations


• Whole Team Ownership of “Quality”• Building it ‘Right’; Building the ‘Right’ Thing

• Healthy – Agile Centric Metrics• Center of Excellence or Community of Practice

• Strategic balance across 3 Pillars; Assessment, Recalibration, and Continuous Improvement

17

Foundation of the 3 Pillars

• Whole Team Ownership of “Quality”

• Whole team view includes building it right, everyone tests,

• Focus on features/stories, confirmation, i d i h d• Building it ‘Right’; Building

the ‘Right’ Thing

• Healthy – Agile Centric Metrics

• Center of Excellence or Community of Practice

conversation, and getting them staged properly OVER testing

• 4-style metrics???

• Agile strategies need light-handed “steering”; establish a CoE (heavier weight) or a CoP(lightweight)

• Consider finding an assessment framework


• Strategic balance across 3 Pillars; Assessment, Recalibration, and Continuous Improvement

• Consider finding an assessment framework and then tying it to your strategy measurement, recalibration, and continuous improvement.

• Make the Foundation visible thru Information Radiators and metrics


Development & Test Automation A central part of agile adoption is focusing on CI, 3-

tiered Automation development, and Dashboards to b i i t ll b ildi f f t• Pyramid-based

Strategy: (Unit + Cucumber + Selenium)

• Continuous Integration

• Attack technical infrastructure in the Backlog

begin incrementally building coverage for faster feedback on changes.

In the interim, Hardening or Stabilization Sprints and having a risk-based Release Train concept help

It’s important that Test or QA not ‘own’ the tooling or all of the automation efforts. The strategy can come from Test, but the tactical automation development is best left to the team


• Visual Feedback –Dashboards

• Actively practice ATDD and BDD

best left to the team.

Mature teams invest in automation as part of Done-ness and continually on their backlogs

18


Software Testing

• Risk based testing:

Exploratory Testing (Charter / Session based and paired) can be an incredibly effective way to establish a whole-team, collaborative view towards quality and testing. It also emerges new tests.

• Risk-based testing: Functional & Non-Functional

• Test planning @ Release & Sprint levels

• Exploratory Testing

• Standards – checklists,

Leverage ‘plans’ as a whole-team collaboration mechanism…and do plan.

Do not measure testing or tester progress; instead, measure throughput, output, sprint outcomes, and done-ness escapes at a team level.

You need a balanced test team; not everyone needs to be able to program But everyone needs to be


,templates, repositories

• Balance across manual, exploratory & automation

to be able to program. But everyone needs to be skilled testers.

Agile testing is a Risk-Based play in every Sprint and across a release sequence. Don’t forget your techniques!


Cross-FunctionalTeam Practices

One of the hardest areas to get ‘right’ culturally. It needs leadership alignment from Quality/Testing to Product to Development and a consistent voice of

h l t h• Team-based Pairing

• Stop-the-Line Mindset

• Code Reviews & Standards

• Active Done-Ness

whole-team approaches.

This is where LEAN lives, where whole-team collaboration happens, where professionalism and craftsmanship are held dear.

I like the view of testers becoming the VOC, champions of quality, and consistent questioners of what is being build. Are we solving the right problems as simply as possible Notions of Minimal


• Aggressive Refactoring of Technical Debt

• User Stories – 3 Amigo based Conversations

problems…as simply as possible. Notions of Minimal Viable Product / Feature help with focus.

And yes Virginia, there ARE standards, templates, and a focus on consistency!

19

Software TestingStrategies

It ALL starts with empowering testers AND creating a Whole-Team view towards QualityWhole Team view towards Quality

Critical Early Steps:Creating a sense of empowered Functional TeamApplying Testing Standards across all teamsDeploying Exploratory Testing across all teamsDefining a core set of Agile KPI / metrics


Defining a core set of Agile KPI / metricsACTIVE participants in Sprint Planning

Cross-Functional Team PracticesStrategies

Training Agile / Lean in general Story writing Acceptance Unit testingAgile / Lean in general, Story writing, Acceptance, Unit testing, etc.Teaming – for example: feedback or 5 Dysfunctions / Trust

Critical Early Steps:Coaches & Scrum Masters to reinforce: Pairing / Swarming; WIP Limits across teamsDefine prescriptive and aggressive Done-Ness for ALL teams


Implement coding standards & Crucible / code reviews across the center (appropriate for technology stacks)Release Planning BEFORE allowing a team to start Sprint #1Backlogs have Bug + Refactoring + Automation targets (20%)?

20

Organizational Quality Strategies

Continuously communicate your unified Vision

Your strategy must be aligned/shared across:Development, Quality/Testing, and Product

Keep working your strategy across the pillarsDon’t get stuck with too narrow a focus (easy road)

Make your strategy visible (Information Radiators)Show progress (Ex: burn up of test automation coverage…across tiers)

Vi li i ti l i di t t A il Q lit


Visualize organizational impediments to your Agile Quality strategies

Attack them!Quarterly read-outs on progress, plans and adjustments

Listen to your teams; Celebrate successes!

What will be (your) agile strategy when you get back home?

Either in groups or individuallyConsider the 3 Pillars discussionConsider the 3-Pillars discussionConsider your current team / organization agile ‘state’

Define a broad, 3-pillar view towards some immediate focus points when you get back into the office

What will you focus on? Why?How will you communicate the need for change?How will you measure results?What will come immediately afterwards?


21

Wrapping up…

Agile is the best thing that’s happened to testers since…ppThe Great Depression

Whole Team viewTesting, Metrics, AutomationPlanning, Reporting, Quality

Facilitate feedbackMulti-tiered automationJust-in-Time risk-based testingJust in Time, risk based testingContinuous improvementTrust the Team

Retrospective


Contact Info

Bob GalenPrincipal Consultant,

RGalen Consulting Group, L.L.C.

Experience-driven agile focused training, coaching & consulting

Cell: (919) [email protected] www.rgalen.com

[email protected] www.velocitypartners.net

BlogsBlogsProject Times - http://www.projecttimes.com/robert-galen/BA Times - http://www.batimes.com/robert-galen/

Podcast on all things ‘agile’ - http://www.meta-cast.com/

44Copyright © 2014 RGCG, LLC 44

22

Additional Topics


Two Pillars of Lean ‘Thinking’

Respect forP l

Continuous ImprovementPeople

Customer, Employees, Vendors…Develop your teamsTrust & coachN t f l k

Improvement

Embrace change, challenge everythingKaizen – small, incremental changeKaikaku – larger scale,No wasteful work Kaikaku larger scale, fundamental

46

From http://www.leanprimer.com


23

Agile Testing QuadrantsBrian Marick; Lisa Crispin & Janet Gregory

Exploratory testingS i

Functional testsSt t t

Automated & Manual

ManualBusiness Facing

ScenariosUsability testing

UATAlpha / Beta

Unit testsComponent tests

Story testsExamplesPrototypesSimulations

Performance testingLoad testing

Q1

Q2 Q3

Q4

Sup

porti

ng th

e Te

amC

ritique the Product


Component testsAPI tests

Load testingSecurity testing

Non-functional requirements

Automated & Manual

Automation, Tools, and

ManualTechnology Facing

10 Tenets of Agile Testing Jean Tabaka, Rally Software

1. The system always runs

2. Stop the line, vs. logging


Lean – fix it now!p gg gdefects

3. If it’s not tested, it’s not “Done”

4. Testing comes first, not last

Early feedback; Earned Value

Collaborative testing, focus on building in quality


5. Finding defects after Development is “Done” is too late

g q y

Early feedback; fix it now!

24

10 Tenets of Agile Testing Jean Tabaka, Rally Software

6. “Development Complete” is meaningless

Whole Team complete view – no “partial credit”

7. Use testing, not analysis, to explore requirements

8. Automation is “how” not a “whether” or “when”

9 Tests are your second most

Executable requirements

Automate all testing; feedback

Code is first; later is


9. Tests are your second most detailed specification

10. Testers are Customer-Developer liaisons

traditional specifications

VOC; guide effective team collaboration; ask the right questions

10 Commitments of Agile Testing Jean Tabaka, Rally Software

1. We commit to not moving forward if a hole is found through root cause analysis without first writing a testthrough root cause analysis without first writing a test

2. We commit to not relying solely on just automated testing or just manual testing

3. We commit to not sitting behind a QA wall (no boundaries!)

4. We commit to not allowing a code complete without test code harness complete


test code harness complete5. We commit to not waiting for a test phase but rather

working in smaller and smaller pieces, sooner and sooner

25

10 Commitments of Agile Testing Jean Tabaka, Rally Software

6 We commit to not testing one iteration after6. We commit to not testing one iteration after development is “Done”

7. We commit to not allowing surprises to accumulate for large end-to-end testing (“mock it now”)

8. We commit to not leaving the riskiest tests to the end9. We commit to being an equal participant with the


customer and the developer in defining “Doneness”10.We commit to not taking this oath lightly