Embed Size (px)
Citation preview
Understanding the role of testing vis-à-vis other life cycle activities w.r.t quality
Getting an appreciation of the basic principles of testing
Basis Material for this session: Software Testing – Principles and Practices, Srinivasan Desikan and Gopalaswamy Ramesh, Pearson Education, 2006-2007
2
Today’s Agenda
Requirements gathering Design Development Testing Deployment and maintenance
3
Phases of software development (not necessarily
sequential!)
4
Quality during various phases…
Eff
ecti
ven
ess
of
Tes
tin
g p
has
e
LOW
HIGH
Quality of other phases
LOW HIGH
Not sustainable!
High Rework Costs Ideal State!
Risky!
5
The car is complete…You just have to paint it!
6
This car can accelerateFrom 0 to 60 in 5 secs Yeah, but it accelerates
when I press the brake!
A product should meet the customers’ needs Customers need not be “external” customers
Testers are customers for developers A software should be “testable”
7
Principle #1
Testing should find out defects before customer finds them out!
Practice of incorporating customer supplied test cases into battery of tests
Including customers and / or “customer facing” into the testing functions
Concept of Alpha / Beta testing
8
Principle #1:Implications
Remember the “triangle” problem? Remember the last class “6 character
problem”?
9
Principle #2
Dijkstra’s Doctrine:Program Testing can never prove the absence of bugs,
Only their presence
Choose your tests intelligently Choose tests that have a higher probability
of uncovering defects Boundary Values, Choice of regression tests
10
Principle #2:Implications
11
Principle 3:
Correct Reqmts.
Defects in reqmts.
Correct design
Defects in reqmts
Defects in design
Correct code
Defects in reqmts
Defects in design
Defects in code
Defects found
Defects in reqmts
Defects in design
Defects in code
Defects not found
Testing Phase
Coding Phase
Design Phase
RequirementPhase
Testing Phase
Coding Phase
Design Phase
RequirementPhase
Reqmts Design Coding Testing Post release
10x
100x
1000x
12
Principle #3
Defects should be detected and corrected at the earliest –Postponing detection of defects compounds cost!
Different types of testing interspersed during product cycle
Different methods of integration Concept of Regression Testing
13
Principle #3: Implications
14
Principle #4:The Cat and the Saint
Knowing why you are testing something is as important asKnowing what you are testing
White Box Testing Starting from code
Black Box Testing Starting from external functionality
Integration Testing Stuff works together!
System / Performance Testing Works in real life conditions
Regression Testing Makes sure changes don’t affect existing stuff
Domain / adhoc testing Bringing domain expertise into testing
15
Principle #4:Implications
16
Principle #5
Test the tests first!A defective test is even more dangerous than a defective product!
A test should document the Input data expected results the test process
The behavior has to be externally corroborated No Turing Machine can verify itself!
17
Principle #5:Implications
18
Principle #6:The Pesticide Paradox
Bugs are like bacteria – they develop resistance to antibiotics;New antibiotics have to be developed!
Need to redefine and refine tests as we move forward
As we “pass” old tests, new defects will surface
Regression Tests have to be “retired” and new regression tests have to be designed
19
Principle #6:Implications
20
Principle #7:The Rags and The Convoy
Number of defects found already
Def
ects
yet
to
be
fou
nd
21
Principle #7:
Defects come in convoys ; Fixing a defect in the convoy is likely to add more defects
Look for side effects Look for “spaghetti code” or “rags” Tie maintenance and testing functions
closely
22
Principle #7:Implications
23
Principle #8:Cop on the Bridge
Prevention is better than cure –Prevention can open substantially new vistas
Quality Assurance Process oriented Defect Prevention oriented Proactive Staff function
Quality Control Product oriented Defect detection / correction oriented Reactive Line function
24
Prevention and Cure:Quality Assurance Vs Quality Control
Double edged sword!
Excessive Process orientation
Lack of Testing makes defects reach the customers
Not a healthy state!
Lack of standards foster “defect breeding”
Lack of Testing makes defects reach the customers
May be resource intensive but gives better payback
Institutionalizes Quality
Makes Quality visible to customers
Last minute rushes
Higher People dependency
Testers as “heroes” and “adversaries”
Double edged sword!
Excessive Process orientation
Lack of Testing makes defects reach the customers
Not a healthy state!
Lack of standards foster “defect breeding”
Lack of Testing makes defects reach the customers
May be resource intensive but gives better payback
Institutionalizes Quality
Makes Quality visible to customers
Last minute rushes
Higher People dependency
Testers as “heroes” and “adversaries”
Low High
Low
Hig
h
Defect prevention focus
Def
ect d
etec
tion
foc
us
25
Principle #8 leads to Principle #9:
Defect Detection
Defect Prevention
26
Principle #9:End of the Pendulum
Achieve a balance between (Defect Prevention) Quality Assurance and (Defect Detection) Quality Control
View QA and QC as supplementary and not adversary job functions
Distribute QA / QC functions throughout for early detection (Principle #1)
27
Principle #8 and #9:Implications
The IBM experiment (Peopleware, DeMarco & Lister)
Issues of “pride” we discussed last class
28
Principle #10:The Men in Black
Instilling pride in testers Establishing an identity for testers Showing them a career path
29
Principle #10:Implications
30
Principle #11:Automation Syndrome
Focus on WHAT you want to automate;Relate it to the goals;
Automation is not a panacea
#1: Goal of testing is to find defects BEFORE customers find them
#2: Program testing can only show presence of bugs, never their absence
#3: Test early and often #4: Understand the “why” and not just the “what” / “how” #5: Test the tests first #6: Tests develop immunity – discover new tests! #7: Focus on convoys of defects #8: Testing encompasses defect prevention #9: Strike a fine balance between defect prevention and
defect detection #10: Plan the automation initiatives #11: Instill pride in testers
31
Putting it all together…
Understanding the role of testing vis-à-vis other life cycle activities w.r.t quality
Getting an appreciation of the basic principles of testing
32
Today’s Agenda Recap
