Finding the important bugs- A talk by John Scarborough, Director of Testing, Aditi Technologies

O P E N TA L K S E R I E S

P R E S E N T S

Be part of the learning experience at Aditi.

Join

the talks.

Its not

training. Its

mind-opener.

Speak at these

events. Or bring

an expert/friend

to talk.

www.aditi.com2

How to enjoy an Open Talk

3

BRING COFFEE & FRIENDS SWITCH OFF MOBILE SWITCH ON MIND SIGN THE ATTENDANCE SHEET

SHARE YOUR WISDOM QUESTION NOTIONS THANK THE TALKER SPREAD THE WORD

www.aditi.com

But did you findthe important bugs?John Scarborough

Director of Test

4 www.aditi.com

Roots, Trunk, and Leaves: The Inherent Hierarchy

5

BLOCKING

BUGS

MASKING

BUGS

FAULTS

FAILURES

BUG REPORTS

www.aditi.com

What Makes Bugs Important?

6

Quality

Customer Values

Speed

Cost

QUALITY DIMENSIONS

Functionality

Scalability

Security

Accuracy

Performance

Maintainability

Interoperability

www.aditi.com

What is CoQ?CoQ is what a company or an organization spends in preventing, testing for, and recovering from

defects.

7 www.aditi.com

Quality doesn’t start with testing

8

BETTER QUALITY

FASTER TIME TO MARKET

CHEAPERCOST

• You can lower your costs and reduce time to market,

but quality will decline.

• You can raise your quality bar and lower your cost, but

time to market will increase.

• You can raise your quality bar and reduce time to

market, but your costs will rise.

Everyone wants Better, Faster, and Cheaper, but to get more

than two out of three, you need to invest in quality before

testing begins.

www.aditi.com

Total CoQ

9

COST OF

QUALITY

Defect

Prevention

Recovery

from failure,

pre-release

Testing

Recovery

from failure,

post-release

www.aditi.com

1. Defect Prevention

10

• Developer training

• Right Dev tools for the right job

• Requirements analysis*

• Clear specification

• Testability analysis*

• Fault-tolerant design

• Defensive programming

• Early prototyping

• Usability analysis*

• Accurate internal

documentation*

All of these can contribute (or if

ignored may fail to contribute)

to the initial quality of the

software under test

* May (rarely) be assigned to QA

COST OF QUALITY

Defect prevention

Recovery from failure, pre-release

TestingRecovery

from failure, post-release

www.aditi.com

2. Testing

11

• Static analysis

• Test environment

correctness

• Test case design

• Test case

maintenance

• Test automation

development

(minimum: BVTs)

• Test automation

maintenance

Software testing and quality engineering

(STQE) can’t test quality into software any

more than you can measure height into a

basketball player.

But testing can reduce CoQ if used

intelligently. The emphasis must be on

discovering defects as early in the SDLC as

possible.

COST OF QUALITY

Defect prevention

Recovery from failure, pre-release

TestingRecovery

from failure, post-release

• Design review

• Code inspection

• Unit testing

• Glass-box testing

• Black-box testing

• Bug-fix testing

• NFR (Non-functional

testing)

• UAT

• Beta testing

• Training testers

STATIC ASPECTS DYNAMIC ASPECTS

www.aditi.com

3. Costs of failure, pre-release

12

• Fixing bugs

• Testing bug-fixes (regression)

• Testing for collateral damage

• Opportunity cost for Dev & Test

• Wasted marketing time

• Direct cost of late shipment

• Opportunity cost of late

shipment

COST OF QUALITY

Defect prevention

Recovery from failure, pre-release

TestingRecovery

from failure, post-release

Fixing bugs one or two phases

later in the SDLC costs 5-75

times as much as fixing them

in the phase in which they

were introduced.

Testing is not the only

defense against faulty

software, but it is certainly

the last.

www.aditi.com

4. Costs of Failure, Post-Release

13

• Support costs

• Root-cause analysis per ticket

• Refunds and recalls

• Interim bug fix releases

• Shipping updated product

• Supporting multiple versions

in the field

• Lost sales

• Lost customer goodwill

Fixing bugs after release

costs 100+ times as much

as fixing them when they

are introduced.

COST OF

QUALITY

Defect

prevention

Recovery

from

failure, pre-

release

Testing

Recovery

from

failure,

post-

release

www.aditi.com

What makes bugs important (or unimportant)…

14

WILL DEVELOPMENT

FIX THEM?

IF NOT, WHY?

www.aditi.com

…which is related to…

15

PRIORITIZATION BASED ON

BUSINESS PURPOSE

www.aditi.com

Risk is always important…

16

RISK• FREQUENCY OF USE

• RICKETY CODE

• REGRESSION

• COLLATERAL DAMAGE

www.aditi.com

…and so is the customer

17

USER EXPERIENCE• MISSING FEATURES

• EXCESSIVE DELAYS

www.aditi.com

Test, Evaluate, Act

18 www.aditi.com

THANK YOU

John Scarborough - Director of Test