View
219
Download
0
Tags:
Embed Size (px)
Citation preview
1
Software Testing and Quality Assurance
Lecture 14 - Planning for Testing (Chapter 3, A Practical Guide to Testing Object-
Oriented Software)
2
Lecture Outline To learn how to analyze the risks
associated with verifying the required functionality.
Adequacy of test cases.
3
Risk analysis ―A tool for testing Risk analysis is part of planning and
development effort. A risk ―anything that threatens the
successful achievement of a project’s goals
A risk is an event that has some probability of happening and, if it occurs, there will be some loss (down time, financial).
4
Risk analysis ―A tool for testing
Fundamental principle for risk-based testing Test most heavily those portions of the
system that pose the highest risks to the project to ensure that the most harmful faults are identified.
5
Risk analysis ―A tool for testing: risk types
Project risks: include managerial and environmental risks (e.g. insufficient supply of qualified personnel).
Business risks: are associated with domain-related concepts. This type of risk is related to the functionality of the program (e.g. changes on the health insurance policy).
6
Risk analysis ―A tool for testing: risk types
Technical risks: include some implementation concepts (e.g. the quality of the code).
7
Risk analysis ―A tool for testing: risk analysis Risk analysis ―is a procedure for identifying
risks and for identifying ways to prevent problems from becoming real.
The output of risk analysis is a list of identified risks in the order of the level of risk that can be used to allocate limited resources and
to prioritize decisions.
8
Risk analysis ―A tool for testing: risk analysis Risks in OO software projects are unique to
the architectural features: Complex interactions among objects Complex behavior associated with a class
specification Changing or evolving project requirement Complexity of a class measured by the size of its
specification The number of relationships a class has with other
classes
9
Risk analysis ―A tool for testing: risk analysis (cont...)
Source of risks: For system testing, the various uses of the
system are prioritized based on the importance to the user and proper operation of the system.
Risks are also associated with the programming language and development tools that are being used to implement the software (strong typed vs. weak typed languages).
10
Risk analysis ―A tool for testing: risk analysis (cont...)
Conducting the analysis: Risk analysis technique includes three
tasks: Identify the risks that each use case poses to
the development effort (classify as low, medium and high –may be also very high)
Quantify the risk Produce a ranked list of use cases
11
Risk analysis example Brickles example
You may combine the frequency and the criticality to determine which should be tested more heavily.
Use Risk Level
Frequency
Criticality
Scenario
Wins Medium Low High Player wins game
Loses
Medium High Low Player loses game
12
Risk analysis example Risk level:
Conservative strategy: select the higher value.
Averaging strategy: select the value between the two values.
13
A testing process: planning issues Issues that must be addressed to give a
basic shape to the test process: Planning issues:
Testing the models. Class testing, interaction testing, system
testing, regression testing.
14
A testing process: dimensions of software testing Who performs the testing?
Developers, independent testers or combination of the two.
Developers may exchange code and test each other’s code.
Which pieces will be tested? Test nothing, test a sample, test
everything, or just the ones associated with high risks.
15
A testing process: dimensions of software testing
When will testing be performed? Test every day, test components as they
are developed, test all components together at the end, at special milestones.
How will testing be performed? Knowledge of specification only,
knowledge of specification and implementation.
16
A testing process: dimensions of software testing (cont...)
How much testing is adequate? No testing, exhaustive testing (consider the
expected lifetime of the software), is the software life-critical,
Coverage: is a measure of how completely a test suite exercises the capabilities of a piece of software (other measures: if every LOC executed at least once, the number of requirement that is checked by the test suite).
17
A testing process: adequacy of test cases
Adequacy of test cases: test the software enough to be reasonably sure that the software works as it is supposed to.
18
A testing process: adequacy of test cases
Adequacy can be measured based on coverage: Coverage based on the requirements: based on
what the software is supposed to do ―how many of the requirements called out in the specification are tested
Coverage based on the code: based on how the software actually works ―how much of the software was executed as a result of running test suites
19
A testing process: adequacy of test cases (cont...)
Functional testing (specification-based or black box testing): test cases are constructed based solely on the
software’s specification and not on how the software is implemented.
Structural testing (implementation-based or white box testing): test cases are constructed based on the code
that implements the software.
20
A testing process: adequacy of test cases (cont...)
Use some combination of both approaches.
21
A testing process: adequacy of test cases― risk analysis (cont...)
Risk analysis in the testing process is applied to determine the level of details and amount of time to dedicate to testing a component.
A reasonable scale of increasing risk for components is: Prototype components Production components Library components Framework components
Ris
k in
crea
ses
22
Key points A risk ―anything that threatens the
successful achievement of a project’s goals Risk analysis technique includes three tasks:
identify, quantify the risk and produce a ranked list of use cases.
Adequacy of test cases: test the software enough to be reasonably sure that the software works as it is supposed to.