Upload
kerry-cook
View
255
Download
3
Tags:
Embed Size (px)
Citation preview
• Introduction to Software Testing• Types of Software Testing• Unit Testing Strategies– Equivalence Class Testing– Boundary Value Testing– Output Testing
• System Testing• Usability Testing
Outline
2
• software testing: Evaluating software by observing its execution
• Primary goal of testing is to find bugs!• During testing, should execute tests that find
bugs.• “Testing can only prove the presence of bugs,
not their absence.” - Dijkstra • When do you stop testing?
What is Software Testing?
4
• test case: A set of inputs, execution conditions, and a pass/fail criterion.– Example: height input is 5, width input is “Hello”, the
test passes if the program produces an error message.
• test or test execution: The activity of executing test cases and evaluating their results.
• test suite: A set of test cases.
Test Terminology
5
• defect: A mistake in the source code (static)– Synonyms: fault, bug
• failure: An observable incorrect program behavior (output)
• debugging: The process of finding and correcting a defect in a program given a failure
More Test Terminology
6
// Solve ax2 + bx + c = 0struct RootAnswer { float root_one; float root_two;}RootAnswer roots(float a, float b, float c);
• Which values for a, b, c should we test?
• Need to select test cases intelligently!
Testing Inputs
8
• Due to time constraints, only a small subset of actual inputs can be used to test.
• Need to choose a test suite such that…– Each “functionality” provided by a program is
tested by at least one test case.– Tests target common programming mistakes.– Tests exercise areas of code that are likely to be
“buggy”.
Test Case Selection
9
• Code Inspection: Looking at the source code directly for bugs.
• Tools– compiler– static analyzer– memory leak detector– profiler / performance monitors
Other Techniques for Finding Bugs
10
• Introduction to Software Testing• Types of Software Testing• Unit Testing Strategies– Equivalence Class Testing– Boundary Value Testing– Output Testing
• System Testing• Usability Testing
Outline
11
• Functional testing: Testing to determine if the software is working properly based on the specification (has the proper function).
• Non-functional testing: Testing other desirable properties of the software.– Examples: performance, usability– Testing is vastly different.
Functional vs. Non-Functional Testing
12
• Unit Testing: Testing of a “unit” in the program.– Unit: smallest part of the program – only assigned
to one developer.• Often a single function or class.
– Focuses on the functionality of that unit.– Can be automated using unit testing tools• Visual Studio• Junit• Google Testing Framework
Unit Testing
13
• Integration Testing: Testing the interactions of the different units.– Assumes unit testing is complete or will catch
errors within the unit.– Focuses on the interactions between units.– Can be tested manually or can be automated.
Integration Testing
14
• System Testing: Testing the entire system.– Requires the entire system to be present.– Used to test common usage scenarios.– Used to test non-functional requirements.– Best tested manually.• Testing approaches are vastly different for non-
functional requirements.
System Testing
15
• Introduction to Software Testing• Types of Software Testing• Unit Testing Strategies– Equivalence Class Testing– Boundary Value Testing– Output Testing
• System Testing• Usability Testing
Outline
16
• Derive tests from the specification of the unit.• Need to choose tests…– Intelligently– Systematically– Thoroughly
• Consider the unit under a test as a “function” with inputs and outputs– All unique and interesting inputs should be tested– All unique and interesting outputs should be produced– Focus on common errors
Unit Testing Strategies
17
• Introduction to Software Testing• Types of Software Testing• Unit Testing Strategies– Equivalence Class Testing– Boundary Value Testing– Output Testing
• System Testing• Usability Testing
Outline
18
• Equivalence Class Testing: Partition the input space into equivalence classes.
• The key is to divide the input space into equivalence classes such that all inputs within each equivalence class have the same functionality.– Very likely that each input within a equivalence class will
exercise the same code within the function.
• Need to have at least one test in each equivalence class.– More tests are better.
Equivalence Class Testing
19
Equivalence Class Testing
Failure (valuable test case)Failures are sparse in the space of possible
inputs ...
... but dense in some parts of the space
If we systematically test some cases from each part, we will
include the dense parts
Functional testing is one way of drawing orange lines to isolate
regions with likely failures
The
spac
e of
pos
sibl
e in
put v
alue
s
No failure
20
• The division into equivalence classes is based on the desired / specified functionality of the subject under test.
• Equivalence classes may consist of:– one particular value– a range of values– a subset (possibly discontinuous) set of values
• The set of equivalence classes should include both valid and invalid inputs.
Equivalence Class Testing
21
1. Volume discount if you buy more than 1000 units.
2. Function that returns the number of days in a month (1-12).
3. If the area code is 800 or 888, there is no charge for the call.
4. Shipping rates differ depending on what country you are in.
Equivalence Class Testing Examples
22
Identify equivalence classes to test for each for a function that returns a letter grade (A, B, C, D, or F) based on a numeric score from 0-100.
Class Problem: Equivalence Class Testing
23
• Introduction to Software Testing• Types of Software Testing• Unit Testing Strategies– Equivalence Class Testing– Boundary Value Testing– Output Testing
• System Testing• Usability Testing
Outline
24
• Boundary Value Testing: Test the values near and at the boundary of an ordered input.
• For ordered inputs, history suggests that values near the boundary are most error prone.– Programmers may forget to check for a boundary condition– Programmers may code the boundary incorrectly (use < instead
of <=)– Special case or different code may be used at the boundary
Boundary Value Testing
Possible test case
25
• For each boundary n, want to have three cases:n – 1, n, n +1
• For a range of values from x to y, have at least six cases:x – 1, x, x + 1, y – 1, y , y + 1
• Boundary value testing can be combined with equivalence class testing:– Boundaries mark the endpoints of equivalence classes.– Need to test the boundaries of each equivalence class.– Still desirable to test one or more values in the middle
of the range.
Boundary Value Testing
26
• Some values are outside of the range:– May also be part of another equivalence class
(worry about redundancy later)– May be invalid – should check to make sure it’s
invalid• Boundary analysis can be applied to more
situations than handling numeric data:– string length– size of arrays (or other data structures)– data type limitations (size of integer)
Boundary Value Testing
27
Class Problem: Boundary Value Testing
28
Identify boundary values to test for each for a function that returns a letter grade (A, B, C, D, or F) based on a numeric score from 0-100.
• Introduction to Software Testing• Types of Software Testing• Unit Testing Strategies– Equivalence Class Testing– Boundary Value Testing– Output Testing
• System Testing• Usability Testing
Outline
29
• Equivalence class and boundary value testing can also be applied to the output of a function.
• Outputs can be divided into equivalence classes.– At least one test should be executed for each output
equivalence class.– In many cases, but not all, tests created for output are
redundant to equivalence class tests created for inputs.
• Boundaries of an ordered output should also be tested.
Output Testing
30
• Introduction to Software Testing• Types of Software Testing• Unit Testing Strategies– Equivalence Class Testing– Boundary Value Testing– Output Testing
• System Testing• Usability Testing
Outline
31
• System Testing focuses on the whole system:– Consists of not only software, but possibly
hardware and other components. – Necessary to understand the solution / product as
a whole.• Encompasses functional and non-functional
testing:– Functional: Does the software work?– Non-functional: Performance, usability, security,
etc.
System Testing
32
• Scenario-Based Testing: Develop a scenario or a script that is important to execute.
• Scenarios tend to mimic realistic use of the system. – How would a normal user use the software?
• Include invalid input scenarios:– How would a user accidentally or intentionally
misuse the system?
Scenario-Based Testing
33
• Exploratory Testing: Test the program without constraint.– Also called ad-hoc testing.– No script or scenario.
• Allows the tester to creatively try tests as they explore the application.
Exploratory Testing
34
• Introduction to Software Testing• Types of Software Testing• Unit Testing Strategies– Equivalence Class Testing– Boundary Value Testing– Output Testing
• System Testing• Usability Testing
Outline
35
• Usability Testing: Tests the usability of a software product.
• Typically done with representative samples of real users to try out the software.
• Objective criteria:– Time and number of operations to perform a task– Frequency of user error• blame user errors on the product!
Usability Testing
37
• Ask questions with the user in mind:– Does the user interface make sense?– Are the prompts / messages clear?– Is the response appropriate when an invalid input
is entered?– Is the program intuitive?
• May want to have someone outside the class try out your programs.– Don’t let them read the specifications.
Basic Usability Features
38