24
Lecture Roger Sutton [email protected] CO331 Visual Programming 20: Testing 1

Lecture Roger Sutton [email protected] CO331 Visual Programming 20: Testing 1

Embed Size (px)

Citation preview

1

Lecture

Roger Sutton [email protected]

CO331 Visual Programming

20: Testing

CO331 Visual Programming

Testing

Generally the rate of error detection is proportional to the number of errors still remaining in the software. Consequently the number of errors presents declines exponentially over time.

2

No. of errors

Time0

Testing often continues until it becomes uneconomic. Hence often bugs remain in software.

CO331 Visual Programming

Testing – cont’d

• Purpose – to test that every part of a program works correctly according to the program specification and to engender confidence in the correctness of the program

The task of ensuring that a program does what is expected is called verification. The aim is to verify that a program meets is specification.

The task of showing that the software solves the required problem is called validation.

• Objective – to show the presence of errors in a program. A non-trivial program is unlikely to be 100% error free. Testing involves the design of test data, execution of the program with the test data and evaluation of the results obtained.

3

CO331 Visual Programming

Techniques

• Black box (functional): focuses on the range inputs and outputs of the program ignoring the actual construction of the code.

• White box (structural): tests each and every possible path through software according to the statements and logic of the code.

• Reviews or walkthroughs: desk checking, ‘dry run’• Stepping through code with debugger• Formal methods: program is specified mathematically and proved

logically to meet specification – thus verification but not necessarily validation.

Small programs comprising a single class are usually tested all at once.Larger programs are tested class by class, called unit testing, and then brought together to test the whole, called integration or system testing.

4

CO331 Visual Programming

Interface design

When software components or modules are to collaborate to fulfill a task, then the interface between them must be clear and well-defined. Here interface is understood to be that what is known about the class and hence may be relied on by other classes - it does not rely on how a particular class is implemented.If two modules are to be developed independently, possibly simultaneously, the interface must be defined before work can commence. Invariably this amounts to identifying that information that constitutes the class documentation.This clearly refers only to what is to be made public and hence there may be other methods and fields, declared private, which would not be referenced.

5

CO331 Visual Programming

Documentation

Class documentation usually comprises:• class name• description of the purpose of the class• any version number• author’s name• documentation for each constructor and methodThe constructor/method documentation includes:• method name• return type• parameter names and types• description of purpose and function of method• description of each parameter• description of the value returned

6

CO331 Visual Programming

Understanding code

Often a programmer will be tasked to correct or modify some existing code. Understanding another person’s code often proves challenging if it has not been written in a structured way. Understanding will be facilitated if the style used is consistent and adequate annotation has been included.Good style involves providing

• Multi-line comment at start to indicate purpose of class together with author and version number

• Each method preceded by comment indicating its purpose, parameters and return type

• Consistent use of white space and indentation to indicate levels of nested blocks and control structures

• Meaningful variable and method names• Additional comments as appropriate to aid interpretation

7

CO331 Visual Programming

Manual walkthroughs

Now that program compilation and running is performed at the click of a button, manual walkthroughs are rarely used; possibly considered ‘low-tech’. Even so they can prove very useful.Manual walkthroughs involve

• Printing out the code of classes being tested• Moving away from the computer• Manually tracing the flow of control through the program and noting

the values the variables assume in the process and how they interact with each other

This amounts to a pencil and paper simulation of what happens when the program is run.Clearly as the number of lines of code increase, this approach may become unmanageable. Hence it tends to be used for small sections of code; often where the logic involved is quite intricate.

8

CO331 Visual Programming

Verbal walkthroughs

Again this is usually only feasible at a high level of abstraction or with a small section of code.The object is to find errors in a program in the course of one person explaining the action of the program, class or method to another. This may work in two ways:

• The person receiving the explanation may spot an error• In course of giving the explanation, the speaker might realise what he

might wish to claim is not in fact valid.The second of these possibilities is the more common and hence well worth attempting even if the audience is unfamiliar with the program components

9

CO331 Visual Programming

Print statements

This is probably the most common technique used to understand and debug programs. They can be used in most languages and easily introduced with any editor. The technique, however, appears to be becoming obsolete with the facilities offer by most IDE debuggers.Typically additional print statements are introduced into a program to provide the developer with information such as:

• which methods have been called• the values of parameter• the order in which methods have been called• the values of local variables and fields at strategic points

There are, however, a number of disadvantages that should be borne in mind:

• It is not always practical to add print statements to every method of a class; hence only effective if right methods have been covered.

10

CO331 Visual Programming

Print statements – cont’d

• Too many print statements may lead to information overload. This offer happens when print statement is placed within a loop

• Having served their usefulness it proves tedious to remove them• Once removed, there always a chance they may be needed again and

they have to be re-inserted. This is overcome by inserting the statements permanently but with their execution governed by the state of some boolean variable. When the boolean variable is set to true the print statements are produced; otherwise they are not.

11

CO331 Visual Programming

Test Plan

comprises a set of test cases. Each test is identified, the purpose of the test is described, the data to be used is identified and the expected outcome noted. Once tests have been performed the results may be cross-referenced with the expected outcomes.Performing a test without any view of the expected outcome is not testing; simply experimenting!

• Test data – needs to be sufficiently representative whilst at the same time being the minimum required to test the program adequately.

• Error detection – results at variance with the expected results. Check expected result. If legitimate error, investigate cause, correct and retest program from the beginning of the test plan – referred to as Regression testing.

12

CO331 Visual Programming

Black box testing

• as do all techniques, requires a precise program specification.• focuses on the functional requirements of the software,

i.e. it enables a set of input conditions to be constructed to exercise all functional requirements of the program.

• attempts to find the following types of error incorrect or missing functions interface errors errors in data structures performance errors initialization and termination errors

13

CO331 Visual Programming

Black box test plan

Construction of test plan involves:1. Identify the inputs and outputs2. Decide on all possible ranges of the inputs and outputs3. Divide the ranges into suitable subranges – Equivalence partitioning4. Describe each subrange and show the value of the input variable(s)

immediately below and above the subrange boundary values. – Boundary value analysis

5. Assemble test cases by structuring combinations of inputs to produce required outputs at boundary points

14

CO331 Visual Programming

Example

Suppose a program must accept the price of an orange, the number of oranges requested and to generate the total cost.

• The price of an orange will be less than £1.00 • The number of oranges will be in the range 1 to 20 inclusive.• A suitable error message should be displayed if either input entry in

invalid.

15

CO331 Visual Programming

Possible user interface

Input: Enter the price of an orange: 0.30Enter the number of oranges: 5

Output:The cost of 5 oranges at £0.30 each is 1.50

Error messages:Warning: The price input is not in the range 0.1 to 0.99. Please click continueand re-enter.Warning: Number of requested oranges is outside allowed range. Please click continue and re-enter.

16

Run

CO331 Visual Programming

Code

17

CO331 Visual Programming

Implementation of Bill

18

CO331 Visual Programming

Test planning

1. Identify the inputs and outputs Number oranges Cost Price Error Message

 2. Decide on all possible ranges of the inputs and outputs

The type and limits of the values input and output (as per specification or assumed) are:Number : Minimum integer Maximum integer

1 20Price: Minimum float Maximum float 0.01 0.99Cost : Minimum Maximum 0.01 19.80Error : message - number error, price error

19

CO331 Visual Programming

E.g. cont’d

3. Divide the ranges into significant subranges Number : | | | | | | IntMin 0 1 20 21 IntMax A B C DPrice: | | | | | | FloatMin 0.00 0.01 0.99 1.00 FloatMax

E F G HCost | | | | 0.01 0.20 0.99 19.80 I J K LError : | |message number error price error M N

20

CO331 Visual Programming

E.g. cont’d

4. Describe each subrange and identify value of variable about the subrange limits.

The test cases identified in the previous sub range diagrams are :

21

Case Description Value A Largest invalid small number 0 B Smallest valid number 1 C Largest valid number 20 D Smallest invalid high number 21 E Largest invalid small price 0.00 F Smallest valid price 0.01 G Largest valid price 0.99 H Smallest invalid high price 1.00 I B * F 0.01 J C * F 0.20 K B * G 0.99 L C * G 19.80 M Number error message N Price error message

CO331 Visual Programming

E.g. cont’d

5. Produce test plan. The test plan must cover all cases identified in step 4.

22

Test number

Cases tested

Actual inputs Expected outputs

Number Price Messages Cost 1 A, E, M,N 0 0. 00 number/price 2 A, F, M 0 0.01 number 3 A, G, M 0 0.99 number 4 A, H, M,N 0 1. 00 number/price 5 B, E, N 1 0 .00 price 6 B, F, I 1 0.01 0.01 7 B, G, K 1 0.99 0.99 8 B, H, N 1 1. 00 price 9 C, E, N 20 0. 00 price

10 C, F, J 20 0.01 0. 20 11 C, G, L 20 0.99 19.80 12 C, H, N 20 1. 00 price 13 D, E, M,N 21 0. 00 number/price

CO331 Visual Programming

Test Harness

To test the method it is necessary to assemble test cases that represent possible inputs and the expected outcomes.The test harness is code that supplies the different test cases and checks that the output is as expected and then displays the result.E.g. To test billTotal requires a function testBill: Input parameters: price (Double), quantity (Integer), total

(Double) Output: BooleanTest:

Further the test function needs to setup the context in order to be able to perform the tests. 23

Function testBill(ByVal price As Double, ByVal quantity As Integer, _ ByVal total As Double) As Boolean Dim aBill As Bill = New Bill(price, quantity) Return aBill.billTotal = total End Function

CO331 Visual Programming

Test Harness - implementation

Dim testCases(,) As Double = {{0, 0, -9999}, ….. , {0, 21, -9999}}Dim testComb() As String = {"AEMN", "AFM", "AGM", "AHMN", "BEN", _ "BFI", "BGK", "BHN", "CEN", "CFJ", "CGL", "CHN", "DEMN"} For i = 0 To 12 testId = testComb(i) If testBill(testCases(i, 0), testCases(i, 1), testCases(i, 2)) Then testId += vbTab & " Pass" Else testId += vbTab & " Fail " noOfFails += 1 End If lstResults.Items.Add("Test " & i & vbTab & testId & vbCrLf) Next lstResults.Items.Add("There were " & noOfFails & " failed tests." & vbCrLf)

24

Run