Autumn 1996 Week 4

SEM510

1. Testing Large Systems.

Needs to be more systematic, since a team is involved and there may be no single person familar with the whole system.

“Top-down” or “bottom-up” testing. Top-down testing starts with the main program (top-level)

and tests this using program stubs, then inserts the lower modules in order.

Bottom-up testing starts with the lowest level modules and tests these using driver programs, then works up to the top-level.

Many systems use a mixture of these approaches with different people testing different modules at the same time.

Autumn 1996 Week 4

SEM510

2. Top-down Testing.

Ideally the modules are tested one at a time. Sometimes it is necessary to test a small number

simultaneously. e.g. 3 procedures, one to open a file, one to write to it, and a third to close it.

The lower level modules are replaced by “stubs” A stub has the same parameters as the real module. It writes a message to say that control has passed

correctly to this module. If results are returned, then known values are set

and supplied to the calling module.

Autumn 1996 Week 4

SEM510

3. Top-down Testing.

1. The main module is tested using stubs for all other modules.

2. Subordinate stubs are replaced by real modules one at a time.

3. Tests are carried out as each real module is inserted.

4. On completion of a set of tests, another stub is replaced with a real module.

5. Regression testing (repeating some or all of the previous tests) may also be carried out to ensure that no new errors have been introduced.

Autumn 1996 Week 4

SEM510

4. Bottom -up Testing.

Drivers must be constructed for the lower level modules. The complete program cannot be tested or shown until all the modules have been tested.

1. Low-level modules are combined into clusters (may have a cluster of one).

2. A driver is written to coordinate input and output.

3. The cluster is tested. 4. Drivers are removed and clusters are combined,

moving up the program structure.

Autumn 1996 Week 4

SEM510

5. Comparison.

In practivce, a combination of these methods is used.

Disadvantages of top-down testing Stubs may be difficult to produce. Output may be difficult to interpret. These are the advantages of bottom-up testing.

Autumn 1996 Week 4

SEM510

6. Error prevention at the Design Stage.

“Error detection” means discovery of an error within a system.

“Passive error detection” - found during execution of system (i.e. it crashes).

“Active error detection” - found as result of a test in the software.

Defensive Designing - each module is written assuming that the other modules contain errors. Hence all data supplied to the module must be checked before it is executed (as far as possible) Can check type and range of values, but not actual value.

Autumn 1996 Week 4

SEM510

7. Error Correction.

1. Build redundancy into the system - where possible add extra features to check the values generated so far. Extreme case, two functionally-equivalent systems produced by two different teams and run in parallel.

Error Correction - what to do if an error is found. Usualy either

the system is halted and information printed. for some errors, the system merely records the

error and proceeds. Error Tolerance - isolation of damage caused by an

error. Applies when system needs to carry on after an error has been detected.

Autumn 1996 Week 4

SEM510

8. Examining a Design.

1. Review - someone else reads the specification and comments.

2. Walkthrough - the designer leads a group of colleagues through a section of design. The others ask questions, comment on possible errors, aspects not covered etc.

3. Inspection - formal evaluation of a written document by a group other than the author. It is examined in detail for faults or violation of standards or requirements.

Autumn 1996 Week 4

SEM510

9. Types of Testing

Requirements Analysis

System Design

Implementation

Installation

Maintenance

SYSTEM TESTING

Tested as a whole

VALIDATION TESTING

Results of requirements analysis compared with software constructed.

INTEGRATION TESTING

construction and verification of program and data structures.

MODULE TESTING

concentrates on each module in turn.

ACCEPTANCE TESTING

tests the system with real data. Both verification and Validation.

Autumn 1996 Week 4

SEM510

10. Testing Large Systems.

There are several different stages of testing, namely module testing, subsystem testing, integegration testing and acceptance testing.

“Alpha testing” - system developer tests sytem with real data in presence of the customer. This continues until the system is considered satisfactory.

“Beta testing” - system is delivered to a number of customers, who use it and report back on problems.

Autumn 1996 Week 4

SEM510

11. Inherent Problems in Software System Design

1. Building large systems is a very complex problem.

2. Developers must understand not only the tools and techniques of software engineering but also details of the application domain.

3. Developers must also deal with an enormous variety of user requirements.

4. As computer systems change (often very rapidly), much has to be re-learnt in order to keep up-to-date.

Autumn 1996 Week 4

SEM510

12. C.A.S.E.

Computer Aided Software Engineering. Term was coined in early 1980s to denote the use of tools

which help in the software development cycle. CASE focusses on requirements analysis and design specification.

“Workbenches” are systems designed to support analysis and design stages. They contain tools for diagram editing system, facilities for design anlysis and checking, data dictionaries, query languages, report generation, forms generation tools and skeleton code generators.

They make it easy to try out prototypes as part of the design of the final system.

Autumn 1996 Week 4

SEM510

13. Z and Formal Methods.

When you are part of a large team, it is very important to have all the modules and interfaces rigorously defined.

The use of a formal language for such definitions reduces the opportunity for misuderstandings.

Z describes what a modules does and not how it does it.

To do this, it provides a rigorous definition of the input data and the output results. If the modules have been correctly defined, with no global variables (and hence no strange side-effects) then this is all that the other modules need to know.

Autumn 1996 Week 4

SEM510

14. Example of Z Specification.

in? out !

in? > = 0 out ! = 3 in?

Powerin?, out ! : Z

in? >= 0

out ! = 3 in?

Consider a very simple example. A module which receives a non-negative integer and calculates the value of 3 raised to that power. A diagrammatic version would be:

This may also appear as a schema.

Autumn 1996 Week 4

SEM510

15. References.

Software Engineering - Sommerville. chapter 9 pp153-168

Software Engineering - Pressman. chapter 9 pp295-306 (less detail).

Z An Introduction to Formal Methods - Diller. especially part I.

Autumn 1996 Week 4

SEM510

16. Z Terminology

In the declaration part of the schema for Power, we hadin?, out : Z

This is equivalent to the two statements:

in? belongs to Z, the set of all integers.out belongs to Z, the set of all integers.

In the predicate part, we had the two formulas

in? >= 0out = 3 in?

Autumn 1996 Week 4

SEM510

17. More Terminology

N = = { n:Z | n>=0 . n} N is defined as a set of objects where { n belongs to

the set of all integers | n is greater than or equal to zero . object is n}

i.e. N is the set of all positive integers.

I suggest you use either the full mathematical terminology or the full English version.

Autumn 1996 Week 4

SEM510

18. Describing Sets.

x is a member of the set X x belongs to X x X

These all have the same meaning. You can use any of them, although the last is the usual mathematical terminology for a Z schema.

Autumn 1996 Week 4

SEM510

19. Operations on Sets.

Union. X U Y is the union of set X and set Y. It contains all elements which are members of either set X or set Y. Duplicates are only included once.

Intersection. X Y is the intersection of the sets X and Y. It contains all elements which occur in both X and Y.

Cartesian Product X x Y. This is the set of all pairs whose first element belongs to X and whose second element belongs to Y.

Autumn 1996 Week 4

SEM510

20. Example of Cartesian Product.

Suppose X is the set of letters {a,b,c,d}and Y is the set of numbers {3 .. 6}

Then XxY is the set of all pairs with a letter from X and a number from Y. e.g. (a,3), (a,4), (a,5), (a,6), (d,6), (d,5), (d,3), (d,4), (c,3), (b,6), (b,3), (c,6), .....

Can you fill in the remaining ones?

Autumn 1996 Week 4

SEM510

21. Consider a telephone system.

Let Person be the set of all people eligible to have use of a telephone.

Let Phone be the set of all possible telephone numbers.

Then we have the Cartesian product

( person, Phone) The actual pairs is the set telephones, which is part

of this Cartesian product. i.e. telephones is contained in Person x Phone

Autumn 1996 Week 4

SEM510

22. Example of a Schema

The mastermind program has a module to generate a code.

CodegenColour : {red, blue, green, yellow}Code4 : Colour x Colour x Colour x Colourout : Code4

out Code4

Autumn 1996 Week 4

SEM510

23. Descriptions of Modules.

Pseudo-code, flowcharts, etc describe how a module achieves its results.

They can be converted into code in a particular language.

There is usually only one correct piece of code for one flow-chart.

Z-specification describes the input, the output and the relation between them.

It makes no attempt to describe how this is achieved. There are usually several equally correct

implementations of a Z-specification.

Autumn 1996 Week 4

SEM510

24. Use of Z specification.

Once a Z specification has been agreed, ANY code which carries out this function is acceptable.

Some may be more efficient than others, and these will be preferred.

Other considerations may apply - e.g if the rest of the program is written in one language, it may be desirable to have this module in the same language.

It is often useful for the module to check that the input data is in the correct format and provide some sort of error indication if it is not.