Access Your Way to Highest Test Team Performance

8/14/2019 Access Your Way to Highest Test Team Performance

1/26

Copyright (c) 2009 LogiGear CorporationAll Rights Reserved. 11

.

Access Your Way to

Highest Test Team

Performance

Michael Hackett


Why change?

Testing is often looked upon by

some people as an unmanageable,unpredictable, unorganized practicewith little structure.

Testers know this is not true.

This debate aside- there is alwaysroom for an improved Test Strategy.As quality professionals ContinuousImprovementis a rallying cry.

The first step in improving ourstrategy and turning our team into ahigher performance team is getting agrasp on where we are now!


2/26


What are we trying tosolve?

We commonly hear questions orcomplaints in development such as:

What is testing doing?

Testing takes too long

Testers have negative attitudes

There can be organization issues suchas:

Customer complaintsThe blame game

Product quality problems

You want to know:

What testing is effective?

Are we testing the right things at the right

time?Do I need a staffing upgrade?

What training does our team need?

How does the Product team value the testeffort?


Objective

In this session my objective is togive you a framework forassessing your team.

- how to plan for an assessment,

- how to execute the assessment and

judge your current performance,- what to do with the information

- how to chart an improvement plantoward higher performance.


3/26


Overview

I will conduct this session based

upon practical experience ofconducting a number ofassessment programs at a varietyof development organizationsdomestically and internationally.

I will use my lessons learnedtohelp you avoid pitfalls and failureof your efforts


Overview

Having decided:

Lets look the issue straight inthe eye. Lets deal with theseissues. We want to improve ourperformance and contribution tothe team.

The goal of doing a test processassessment is to get a clearpicture of what is going on in

testing, the good things, theproblems, possible paths toimprovement.


4/26


What is an Assessment?

Assess:

To sit beside, assist in the office of

a judge.

Merriam-Webster Online Dictionary

A test assessment is a datagathering process. To makedecisions we need data.


Important note

This effort is much larger than test

team.

Issues will arise over:

Who owns quality?

What is goal of testing?

If this assessment is done well youmay step on some toes!


5/26


Standards

There are many standards or

assessment processes that are welldocumented for developmentorganizations.

CMM and ISO9000 are the mostcommonly mentioned.

The process of improving or aguideline for standardized processesis not as well agreed upon for testteams.

They are also expensive, timeconsuming, and for most US basedorganizations- what is the goal ofusing them?


Introduction

In our view, a Test Strategy has 3

components that need to work togetherto produce an effective test effort.

Source: LogiGear developed this model by learning

from Mitchell Levy, Value Framework Institute

SP3


6/26


Introduction

We will focus our assessment on:

People, Process for software developmentand test,

Practice, methods and tools yourteam employs to accomplish thetesting task.


Caution!

As a result of an assessment, yourwork may increase. There may be:

more demands for documentation

more metrics

more responsibility for communication

and visibility into testing.


7/26


Lessons Learned

The process will be doomed if:-no executive sponsorship

-no measurement program built

-no tools to support change

-risk averse culture

-the exercise is an effort to blametesting for project-wide failures

-no commitment about the goal oftesting

-no understanding of testing or qualityassurance across product team

-lack of responsibility for quality


The Assessment Process

Phase 1 Pre-Assessment

PlanningGoal- set expectation, plan project, set

a timeline, firm executivesponsorship.

Action- meet with management ofgroups, layout expectation for resultof process, describe plan, establishtimeline

Result- agreement of expectation andbuy-in of the assessment processand follow-up commitment for

improvement.Deliverable- schedule and project plan.

Hint- make sure team understands theassessment is only the beginning.


8/26


The Assessment ProcessBefore you begin

Get executive buy-in.

Make a schedule and stick to it.

Give a presentation of what you aredoing, why and what you hope to getout of it.

Make a statement of goals or outline of

work as a commitment.


The Assessment ProcessBefore you begin

Make a Scope Document a pre-approval/budget deliverable.

What is in scope and out?

The requirements process?

How much unit testing the developers do?

The Product Management team?The Automation Framework and Program?


9/26



Phase 2- Information

GatheringGoal- Develop interview questions and

surveys which become the backboneof your findings. Gather informationand develop more questions.

Action- gather documentation, developinterview questions, develop testteam survey.

Result- you are ready to begin theassessment.

Deliverable- Development Process

Documentation set complete,Interview questions, Tester Survey.

Hint- you will be collecting muchinformation. Get help! Stayorganized.


Sample Documents toGather

Project/Developers

SDLC Document Engineering Requirements Doc

Test Docs

Test Plan

Template and example

Test Case Template and example

Status Reports

Test Summary Reports


10/26


The Interviews

The Interviews need to cover a

wide range of issues. Forexample:

Development Process

Test Process

Requirements issues

Change ControlAutomation

Tool use

Developer Unit testing

Opinions about the test team from

other groupsExpectation of the test effort

Political Problems

Communication issues


The Interviews

In our assessment work we use arange of issues partially based onTPI developed by Tim Koomen,Martin Pol in:

Test Process Improvement : Step-By-Step Guide to Structured Testing

June 1999, Addison-Wesley


11/26


The Interviews

What we like about TPI is the structurethat enables us to frame thepresentation, discussion andimprovement effort.

However, it is essential to think aboutthe definitions of various attributes

being evaluated. Dont be afraid to useyou own attributes and definitions. Theprocess is not cookie-cutterand needsto be customized to cover all basesduring the assessment.

It is essential to formulate targetedquestions particular to tools andtechnologies so you get the informationyou want.


Sample InterviewQuestions

One topic

The Development Process What do you think about the overall

development process?

Do you think the processes havebeen followed effectively?

Have people been trained in theprocess?

What is your opinion of the process?

How is the process constructed?

What do you think is theeffectiveness of the overall quality

process, including, requirementstracking, unit testing, buildengineering?


12/26


The Survey

In addition to the interviews, thesurvey and the doc review are themain sources of informationgathering we do.

Short answer, multiple choice,true/false is easiest to process.

These surveys go to the test teamonly. You are trying to discoverhow much knowledgethe teamhas in a few areas, such as:

Domain of the product

Technical knowledge

Software development lifecycle &process

Test/QA knowledge

Quality theory, test methods


Sample SurveyQuestions

What are the strengths andweaknesses of your group?

How do you think other groupswould characterize the competencyof your group? What makes youdraw that conclusion?

What are the MOST effectivethings in your current approach totesting?

What are the LEAST effectivethings in your current approach totesting?

What would you like to CHANGEor IMPROVE if you could (pleaselist at least 3 items)?


13/26


Pre- Assessment

Just before the Assessment

begins you may want to havea meeting with all staff thatwill be interviewed andsurveyed so that they willunderstand the process and

know you are sampling from awide group. You may want totalk about confidentiality

here covering this may be a

key to success of failure ofthe assessment.



Phase 3- AssessmentGoal- Conduct interviews and develop

preliminary findings

Action- gather and reviewdocumentation, conduct interviews,

send out and collect surveysResult- you build a mountain ofmaterial and information to review.


14/26


The Interviews

The interview process is the mostvaluable aspect of the assessment.

In my experience the interviewquestions do not all have to beanswered- they may be but they alsomay be jumping off points fordiscussions or issues that are on the

mind of the interviewee.

Keep it under 90 minutes.

Hint: Since it is your job to take notesand facilitate discussion and listen

and perhaps ask more questions- setaside time between scheduledinterviews to synthesize, take notesand catch your breath.


The Interviews

Who you choose to interview:

Be careful not to pick only friends or

people who will tell you what youwant to hear.

Pick the widest range possible- thelongest term employees, the newest,the biggest complainer, the quietest,

the least techie, the most tech. Dont interview too many people but

you make sure you get a great mix orpeople from inside and outside thetest group.

This process does not only work to

focus on testers.


15/26



This job can be very large. Criticalreview of the documentation,

interviews, review of surveys is a fulltime job if the project is to be done ina timely fashion.



Phase 4- PostAssessment

Goal- Synthesize information into a listof findings

Action- Review, collate, think, formopinions, make postulations.

Result- You have taken all theinformation gathered from the

document review, interviews andsurvey and developed a list offindings.

Hint-get help.

Deliverable- List of Findings, Collated

Survey answers, Collated Interviewresponses, Staff Assessment, TestGroup Maturity Ranking


16/26


17/26


The Assessment ProcessPost-Assessment

Findings can be categorized into:

PeopleTechnical Skill

Interpersonal skills

Process

Documentation

Test Process

SDLC

Practice

Strategy

Automation

Environment

Tools

And more subcategories that suit your need.


Staff Assessment

Rank of Test Team based upon ourown assessment scale

(2 sample categories):

III. Mathematical Skills

Understand concepts behind data processing algorithm

Apply mathematical operations to such tasks as frequency distribution

Apply statistical analysis techniques

Determine testability, reliability and validity

IV. Reasoning Skills

Define problems

Collect, interpret and analyze data

Establish facts and draw reasonable if not valid conclusions

Interpret complex technical instructions in mathematical or diagram form

Deal with several abstract and concrete variables

Trouble-shoot logically and reasonably

0 = No Knowledge 1 = Some Knowledge R = Required

2 = Knowledgeable 3 = Lead Expert ise D = Des irable


18/26


TPIhas twenty areas to rank.Here are an example three ofthe lifecycle items.

Test Group Maturity



Phase 5- Presentation of

findings with project sponsor,executive sponsor and team.Goal- Present Preliminary findings to

executives and project sponsor. Getagreement on the highest priorityimprovement areas.

Hint- be prepared for a very differentinterpretation of the findings than youperceived.

Deliverable- Improvement Roadmap


19/26


Presentation of Findings

Be prepared for difficult areas.

Before you have a presentation tothe whole team about the results ofyour work, have a review of findingsand roadmap with the executivesponsor to discuss any difficultareas, confidential issues.

Pick the top 5 or top 10 to becomethe catalysts for change or roadmapof actionable improvement.



Phase 6 - Implementation ofRoadmap

Goal- setout goals with timelines andmilestones and subtasks to accomplishthe tasks agreed upon for improvement.

Action- Develop a schedule forimplementation of the improvementplan.

Hint- Get some aspect of the projectimplemented immediately so people cansee tangible results right away- even ifthey are the smallest or easiestimprovement tasks. Roadmap tasksmust also be achievable and budgeted.

Deliverable- The Roadmap forImprovement


20/26


Roadmap Task Example

Title: Integrate better between onshoreand offshore teams to gain higher

productivityPriority: 4

Specific:

Provide training and support to enablestaff on both sides to collaborate moreeffectively

Structure the work environment andrelationship to make it more conducive tosuccess

Provide an effective productivitymanagement framework help facilitateoffshore activities

Solicit feedback and address thefeedback accordingly

Schedule:

Date for completion


Roadmap Task Example

Title: Change/refine test automation strategy,specifically, methods and tools.

Priority: 4Specific:

Improve test automation productivity,failure analysis, scalability, and reducemaintenance costs

Make test automation efforts visible andquantifiable in a meaningful way

Make test automation effortsmanageable

Schedule:

Date for completion


21/26



Month 1 Month 2 Month 3 Month 4 Month 5 Month 6 Month 7 Month 8 Month 9 Month 10 Month 11 Month 12

Task 2:

Test Strategy Workshop

Metric Development and

Improvement Workshop

Task 3:

Final Presentation of Test

Strategy and Execution

Task 4:

Test Planning Design

Analysis Skill Development

Workshop

(3 individual workshops)

Task 1:

Presentation of Findings

Long Term Goal

Implimentation: TBD

Task 5::

Follow-Up Assessment and

Coaching



Once the Assessment andPresentation of Findings iscomplete it is typical to reformulatethe Test Strategy.

One of the most common findings

from an test assessment isdiscovering that the does what itdoes because that is what it hasalways done- not because it iswhat is most effective.


22/26


Testing Strategy in Context

A test strategy is a holistic

plan that starts with a clearunderstanding of the coreobjective of testing, fromwhich we derive a structure for

testing by selecting from manytesting styles and approachesavailable to help us meet ourobjectives.


Test Strategy

Source: LogiGear developed this model by learning

from Mitchell Levy, Value Framework Institute

SP3


23/26


24/26


The Roadmap to change

Metrics Program EvaluationWhile doing the assessment you will,of course, assess the success of themetrics program you have in place.

If you dont have an effective metricsprogram in place now is the time tostart one.

In addition you will want to addmetrics to observe what changes inthe SDLC and the product as a resultof the assessment.


Metrics

If we dont measure we cannever improve!

Anything worth doing is worth

measuring.


25/26


Metrics

Metrics have 2 goals for yourorganization. They are notmutually exclusive.

First- ship an improved productwith a better understanding ofrisk and test coverage.

Second- improve the process sothe next product can be releasemore efficiently with less stressand greater knowledge of the

risk.


Metrics must be Actionable

Project Management- getting the

product out. These are typicallymeasures of software stability andActivity How many test cases done Defect counts Hours tested against a build

Code turmoil Requirements stability

Process Improvement Defect Aging Requirement Stability Valid Defects found vs. test method Bugs by severity post release


26/26


LogiGear Corporation

Contact:Michael Hackett

[email protected]

www.logigear.com