Upload
sreekar127scribd
View
132
Download
3
Tags:
Embed Size (px)
Citation preview
Prof. N. Prasanna Balaji
SOFTWARE TESTING METHODOLOGYTerm:2009-2010
B.Tech III/IT Semester - I
Unit-IV PPT Slides
Text Books: 1.Software Testing Techniques: Boris Beizer 2. Craft of Software Testing: Brain Marrick
2
Sub Topic No’s
Sub Topic name Lecturer No
Slide No’s
1 Introduction L1 3
2 Domains & Paths L2 22
3 Nice Domains L3 28
4 Ugly Domains L4 42
5 Domain Testing L5 51
6 Domains & Interfaces Testing L6 65
7 Domains & Testability L7 70
ref boris beizer 3
Domain Testing - Domains & Paths
Domain Testing ModelDomain Testing Model
INPUT CLASSIFY DO CASE 1 OUTPUT
DO CASE 2
DO CASE 3
DO CASE n
D1
D3
(x,y,z,…)
(1, 2, 3, 4, 5,…)
{ Outcome }
D2
Dn
ref boris beizer 4
Domain Testing Domain TestingDomain Testing
Views Programs as input data classifiers
Verifies if the classification is correct
ref boris beizer 5
Domain Testing - Domains & Paths
Domain Testing ModelDomain Testing Model
Two ViewsTwo Views
• Based on SpecsBased on Specs
• Functional TestingFunctional Testing
• Based on Implementation informationBased on Implementation information
• Structural Technique
ref boris beizer 6
Domain Testing - Domains & Paths
Domain Testing ModelDomain Testing Model
• Variables• Simple combinations of two variables• Numbers from - to +
• Input variables as numbers
• Loop-free Programs
ref boris beizer 7
Domain Testing - Domains & Paths
Domain Testing ModelDomain Testing Model
• Structural Knowledge is not needed. Only Specs.
• For each Input Case,
• A Hypothetical path for Functional testing
• An Actual path for Structural testing
ref boris beizer 8
Domain Testing - Domains & Paths
DomainDomain (simplified) (simplified)
• A Single Connected Set of numbers
• No arbitrary discrete sets
• Defined by the boundaries
• One or more boundaries
• Specified by predicates
• Bugs likely at the boundaries
• One or more variables
D1
ref boris beizer 9
Domain Testing - Domains & Paths
PredicatesPredicates
• Interpretation
• Structural Testing - CFG
• Functional Testing - DFG
• Specifies the Domain Boundary
• n Predicates in Sequence 22n domains
• Or, just two domains A .AND. B .AND. C
ref boris beizer 10
Domain Testing - Domains & Paths
PathsPaths
• At least One PATH thru the Program
• More paths for disconnected domains
ref boris beizer 11
Domain Testing - Domains & Paths
Summary:Summary:
• Domain for a loop-free program corresponds to a set of numbers defined over the input vector
• For every domain, there is at least one path thru the routine, along which that domain’s processing is done
• The set of interpreted predicates traversed on that path (ie., the path’s predicate expression) defines the domain’s boundaries.
ref boris beizer 12
Domain Testing - Domains & Paths - Domain Closure
Boundary: Closed / Open
MIN MAX
D1D2
D3X < MAXX >= MIN
MIN MAX
D1 D2 D3
X >= MIN X <= MAX
Closed
Closed Open
ref boris beizer 13
Domain Testing - Domains & Paths - Domain Closure
Boundary: Closed / Open
MIN MAX
D1 D2 D3
X > MIN X < MAXOpen
ref boris beizer 14
Domain Testing - Domains & Paths - Domain Dimensionality
• One dimension per variable
• At least one predicate• Slices thru previously defined Domain
• Slicing Boundary
• N-spaces are cut by Hyperplanes
ref boris beizer 15
Domain Testing - Domains & Paths – Bug Assumptions
• Processing is OK. Domain definition may be wrong.
Boundaries are wrong.
Predicates are wrong.
• Once input vector is set on the right path, it’s correctly processed.
• More bugs causing domain errors …
ref boris beizer 16
Domain Testing - Domains & Paths – Bug Assumptions..
• Double-zero representation
• Floating point zero check
• Contradictory domains
• Ambiguous domains defined
undefined
ref boris beizer 17
Domain Testing - Domains & Paths – Bug Assumptions..
• Over-specified domains
• Boundary errors
Boundary closure
Shifted
X > 3 .AND. X < 2 .AND. Y > 3
ref boris beizer 18
Domain Testing - Domains & Paths - Bug Assumptions
• Boundary errors….
Tilted
Missing
Extra boundary
ref boris beizer 19
U3
• Closure Reversal
• Faulty Logic
Simplification of compound predicates
X >= 3
Domain Testing - Domains & Paths - Bug Assumptions
ref boris beizer 20
Domain Testing - Domains & Paths - Bug Assumptions
Strategy for domain testing in 2-dimStrategy for domain testing in 2-dim
ref boris beizer 21
Domain Testing - Domains & Paths - Restrictions
In General…
1.1. Coincidental CorrectnessCoincidental Correctness
DT cannot detectExample
• Representative outcomeRepresentative outcome
Partition testing
Input Equivalence
D1
D2
F1(x) : +1
F2(x) : -1
ref boris beizer 22
Domain Testing - Domains & Paths
Domain Testing ModelDomain Testing Model
INPUT CLASSIFY DO CASE 1 OUTPUT
DO CASE 2
DO CASE 3
DO CASE n
D1
D3
(x, y, z, …)
(1, 2, 3, 4, 5,…)
{ Outcome }D2
Dn
Function f1
f2
f3
f4
ref boris beizer 23
Domain Testing - Domains & Paths - - Restrictions
3.3. Simple boundaries & Compound predicatesSimple boundaries & Compound predicates
X0 16
X >=0 .AND. X <= 16
Y = 1 .AND. X >=0 .AND. X <= 16
X0 16
Y
D1
D1
ref boris beizer 24
Domain Testing - Domains & Paths - - Restrictions
Compound predicates….Compound predicates….
• Impact of Impact of .OR..OR.
• Concave, Disconnected
• Adjacent domains with same function
• Example
A B C + D E FA B C + D E F
• Eliminate compound predicates
ref boris beizer 25
Domain Testing - Domains & Paths - Restrictions
4. Functional Homogeneity of Bugs
• Functional form still Retained
a x + b y >= c
• Bugs are only in a, b, c
ref boris beizer 26
Domain Testing - Domains & Paths - - Restrictions
5. Linear Vector Space
• Linear boundary predicate, Interpreted• Simple relational operators
• Conversion to linear vector space
• 2-d Polar co-ordinates
• Polynomials
• Problems with Non-linear Boundaries
ref boris beizer 27
Domain Testing - Domains & Paths - Restrictions
6. Loop-free Software
Predicate for each iteration
Loop over the entire transaction
Definite loop
ref boris beizer 28
Domain Testing - Domains & Paths – Nice & Ugly domains
Nice DomainsNice Domains
RequirementsRequirements
• Bugs ill-defined domains
Before DTBefore DT
• Analyze specs
• Make the Boundary Specs Consistent & Complete
ref boris beizer 29
Domain Testing - Domains & Paths – Nice domains
Implemented Domains
• Complete, Consistent & Process all inputs
Specified Domains
• Incomplete, Inconsistent
Programmer’s / Designer’s Effort
ref boris beizer 30
Domain Testing - Domains & Paths – Nice domains
Nice Domains
• Linear, Complete, Systematic, Orthogonal, Consistently Closed, & Convex
U1 U2
V1
V2
D11 D12
D21 D22
ref boris beizer 31
Domain Testing - Domains & Paths – Nice domains
Nice Domains
Advantages
• Ease of DT
• Fewer Bug Occurrences
ref boris beizer 32
Domain Testing - Domains & Paths – Nice domains
Nice Domains
Boundaries are
1. Linear
2. Complete
3. Systematic
4. Orthogonal
5. Closure consistency
6. Convex
7. Simply connected
ref boris beizer 33
Domain Testing - Domains & Paths – Nice domains
1. Linear Boundaries
Interpreted linear inequalities
n-dim Hyperplane:
n+1 Points
n+1 + 1 Test Cases
Non-Linear
• Transform
ref boris beizer 34
Domain Testing - Domains & Paths – Nice domains
2. Complete Boundaries
• Span the total number space (-, +)
• One set of Tests
Incomplete…
• Reasons
• Tests
ref boris beizer 35
Domain Testing - Domains & Paths - Nice domains
3.3. Systematic BoundariesSystematic Boundaries
• Linear Inequalities differing by a constant
fj (X) ≥ kj or, fj (X) ≥ g (j, c) g (j, c) = j + k * c
• Parallel lines• Identical Sectors in a Circle
• DTDT• Test a domain Tests for other Domains
ref boris beizer 36
Domain Testing - Domains & Paths - Nice domains
4.4. Orthogonal BoundariesOrthogonal Boundaries
• Two boundaries or, boundary sets
• Parallel to axes
• DTDT
• Each Set Independently
• # Tests O (n)
Uj
Vj
ref boris beizer 37
Domain Testing - Domains & Paths - Nice domains
Orthogonal BoundariesOrthogonal Boundaries
• Tilted sets• transformation• Test cases: O(n)
• Concentric circles with radial lines
• Rectangular coordinates• Polar
r ≥ aj .AND. r < aj+1 .AND.
≥ j .AND. < j+1
ref boris beizer 38
Domain Testing - Domains & Paths - Nice domains
Non-Orthogonal BoundariesNon-Orthogonal Boundaries
• Test Intersections O ( n2 ) + O (n)
ref boris beizer 39
Domain Testing - Domains & Paths - Nice domains
5.5. Closure ConsistencyClosure Consistency
• A Simple pattern in all boundary closures
• Example
• Same relational operator for systematic boundaries
D1 D2D3 D4
ref boris beizer 40
Domain Testing - Domains & Paths - Nice domains
6.6. Convex DomainConvex Domain
• Line joining any two points lies with in the domain
• DT• n on-points & 1 off-pt
ConcaveConcave
• “ But, However, Except, Or … “ in Specs• Handle with special care
ref boris beizer 41
Domain Testing - Domains & Paths - Nice domains
7.7. Simply Connected DomainSimply Connected Domain
• In a single piece
• 2 complementary domains
• D1: Convex D2: Concave, not-Connected
• Programmer / Designer
• Convex part First
D1D2
D1D2
D3
ref boris beizer 42
Domain Testing - Domains & Paths – Ugly Domains
Generally,Generally,
From Bad Specs
Programmer / Designer Simplifies => Ugly to Good
possibility of Introduction of more bugs ?!?
ref boris beizer 43
Domain Testing - Domains & Paths – Ugly Domains
CausesCauses
1.1. Non-linear BoundariesNon-linear Boundaries
1.1. Ambiguities & ContradictionsAmbiguities & Contradictions
1.1. Simplifying the topologySimplifying the topology
1.1. Rectifying boundary closuresRectifying boundary closures
ref boris beizer 44
Domain Testing - Domains & Paths – Ugly Domains
1.1. Non-linear BoundariesNon-linear Boundaries
• Transform
ref boris beizer 45
Domain Testing - Domains & Paths – Ugly Domains
2.2. AmbiguitiesAmbiguities
• Holes in input vector space.
• Missing boundary
• Detected by Specification languages & tools.
ref boris beizer 46
Domain Testing - Domains & Paths – Ugly Domains
2.2. Contradictions..Contradictions..
• Overlapping of
• Domain Specs
• Closure Specs
D1 D2
D3
ref boris beizer 47
Domain Testing - Domains & Paths – Ugly Domains
3.3. Simplifying the Topology….Simplifying the Topology….
• Complexity !
• Concavity, Holes, Disconnectedness
ref boris beizer 48
Domain Testing - Domains & Paths – Ugly Domains
3.3. Simplifying the Topology….Simplifying the Topology….
• Smoothing out concavity
• Filling in Holes
ref boris beizer 49
Domain Testing - Domains & Paths – Ugly Domains
3.3. Simplifying the Topology….Simplifying the Topology….
• Joining the pieces
Correct:
• Connect disconnected boundary segments
• Extend boundaries to infinity
ref boris beizer 50
Domain Testing - Domains & Paths – Ugly Domains
4.4. Rectifying Boundary Closures.Rectifying Boundary Closures.
• make closures in one direction for parallel boundaries with closures in both directions
• Force a Bounding Hyperplane to belong to the Domain.
Consistent Direction Inclusion / Exclusion Consistency
ref boris beizer 51
Domain Testing - Domains & Paths – Domain Testing
General DT StrategyGeneral DT Strategy
1. Select test points near the boundaries.
2. Define test strategy for each possible bug related to boundary
3. Test points for a domain useful to test its adjacent domain.
4. Run the tests. By post test analysis determine if any boundaries are faulty & if so how?
5. Run enough tests to verify every boundary of every domain
ref boris beizer 52
Domain Testing - Domains & Paths – Domain Testing
DT for Specific Domain BugsDT for Specific Domain Bugs
Generally,
• Interior point
• Exterior point
• Epsilon neighborhood
• Extreme point
• On point
ref boris beizer 53
Domain Testing - Domains & Paths – Domain Testing
DT for Specific Domain BugsDT for Specific Domain Bugs
Domain D1
Epsilon neighborhood
Boundary point
Extremepoint
On Points
Off Points
ref boris beizer 54
Domain Testing - Domains & Paths – Domain Testing
1.1. 1-d Domains1-d Domains
2.2. 2-d Domains 2-d Domains
3.3. Equality & inequality PredicatesEquality & inequality Predicates
4.4. Random TestingRandom Testing
5.5. Testing n-dimensional DomainsTesting n-dimensional Domains
ref boris beizer 55
Domain Testing - Domains & Paths – Domain Testing
Testing 1-d Domains : Testing 1-d Domains : Bugs with open boundariesBugs with open boundaries
AABB
xx
AABB
xx
AABB
xxx1x1
AABB
xx x1x1
Closure Bug
Shift left Bug
Shift Right Bug
ref boris beizer 56
Domain Testing - Domains & Paths – Domain Testing
AABB
xxMissing Boundary
AABB
xxxx
Extra Boundary AABB
x1x1xx
CCxx xx
Testing 1-d Domains : Testing 1-d Domains : Bugs with open boundariesBugs with open boundaries
Bugs with Closed Boundaries : Similar to the aboveBugs with Closed Boundaries : Similar to the above
ref boris beizer 57
Domain Testing - Domains & Paths – Domain Testing
2.2. Testing 2-d DomainsTesting 2-d Domains
• Closure bug
• Boundary Shift : up / down
• Tilted Boundary
• Extra Boundary
• Missing Boundary
ref boris beizer 58
Domain Testing - Domains & Paths – Domain Testing
Strategy for domain testing in 2-dimStrategy for domain testing in 2-dim
2 n : Domains share tests 3 n : no sharing of tests by domains
ref boris beizer 59
Domain Testing - Domains & Paths – Domain Testing
3.3. Equality & Inequality PredicatesEquality & Inequality Predicates
• An Equality predicate defines a line in 2-d
c’c’cc
cc
AA
BB
aa
dd
bb
To avoid bugs
ref boris beizer 60
Domain Testing - Domains & Paths – Domain Testing
4.4. Random TestingRandom Testing
• A Point in the center : verifies computation
ref boris beizer 61
Domain Testing - Domains & Paths – Domain Testing
5.5. Testing n-Dimensional Domains (strategy)Testing n-Dimensional Domains (strategy)
n-dimensions, p boundary segments
• (n+1)*p test cases : n on points & 1 off point
• Extreme pt shared : 2 * p points
• Equalities over m-dimensions create a subspace of n-m dimensions
• Orthogonal domains with consistent boundary closures, orthogonal to the axes & complete boundaries
• Independent testing
ref boris beizer 62
Domain Testing - Domains & Paths – Domain Testing L5
Procedure for solving for valuesProcedure for solving for values
Simple procedure. Need tools.
1. Identify input variables
2. Identify variables which appear in domain-defining predicates, such as control-flow predicates
ref boris beizer 63
Domain Testing - Domains & Paths – Domain Testing L5
ProcedureProcedure
3. Interpret all domain predicates in terms of input variables:• Transform non-linear to linear• Find data flow path
4. Predicate expression with p # predicates. Find # domains : < 2 p
3. Solve inequalities for extreme points
4. Use extreme points to solve for nearby on points …
ref boris beizer 64
Domain Testing L5
EffectivenessEffectiveness
• Cost effective
• Bugs on boundaries, extreme points
• Hardware logic testing – tool intensive
ref boris beizer 65
Domain Testing L6
Domains & Interface TestingDomains & Interface Testing
• Domain
• Range
Function / Routine
Classify
DomainVariable Range
ref boris beizer 66
Domain Testing L6
Domains & Interface TestingDomains & Interface Testing
• Span compatibility
Routine 2Routine 1Domain
VariableRange for
Routine2
Domain for
Routine2
Range for
Routine1
ref boris beizer 67
Domain Testing L6
Interface Range/Domain Compatibility TestingInterface Range/Domain Compatibility Testing
• Test each var independently
• Find an inverse function
• Build a super domain
ref boris beizer 68
Domain Testing L7
Domains and TestabilityDomains and Testability
• Linearizing transformationsLinearizing transformations
• Polynomial• Rectangular to polar• Generic & rational => Taylor series
ref boris beizer 69
Domain Testing L7
Domains and TestabilityDomains and Testability
Perform transformations for better testing.Perform transformations for better testing.
• Co-ordinate transformationsCo-ordinate transformations
ref boris beizer 70
Domain Testing L7
Domains and TestabilityDomains and Testability
• Canonical Program FormCanonical Program Form
ref boris beizer 71
Domain Testing – Questions from the previous year’s examsU4
1. Explain Nice & Ugly Domains.
2. What is domain testing? Discuss applications of domain testing
3. Explain the domain boundary bugs for two dimensional domains
4. What is the purpose of Domain Testing? Give its schematic representation
5. Define the following concepts.1. Domains 2. Domain closure 3. Domain
dimensionality 4. Bug assumptions for Domain Testing
6. Explain simple domain boundaries & compound predicates.
7. Classify what can go wrong with boundaries, then define a test strategy for each case in domain testing.