1

Software Testing and Quality Assurance

Lecture 34 – Software Quality Assurance

2

Lecture Objectives What is Software Reliability? Classify Failures Software Reliability Engineering

3

What is Software Reliability? Consider two simple systems consisting

of two writers and a reader communicating via a shared memory on a bus.

Suppose probability of failure of each channel of the bus is 10 -5 per operational hour.

4

What is Software Reliability?

5

What is Software Reliability? For a single channel bus

The probability of a bus failure, such as message becoming corrupt or the bus failing to transmit a message is ‘one failure every 11.4 years’.

If we add the second channel then the failure probability decreases to

One failure every 1,114,552 years, because both channels must fail at the same time for this to occur.

6

What is Software Reliability?

7

What is Software Reliability? We have to be careful in not over

interpreting these numbers. While improvement is significant, after all, It is only a probability.

The definition of reliability depends heavily on determining, and in practice detecting, failures.

8

Classifying Failures Examples of failures include

Functional failures – where the actual behavior of the system deviates from the specified functional behavior.

Timing failures – where the program may deliver correct results but the program fails to meet its timing requirements.

Safely failures – where an accident resulting in harm or injury is deemed to be a failure of the system.

9

Classifying Failures In practice, we need

A way of observing failures; and Logging each failure as it arises.

We also need A way of determining the time at which a

failure occurs – called failure time; Or The rate at which failure occur – called

failure rate. Assurance that the estimates are accurate

10

Classifying Failures Transient Failure

Program gives incorrect result, but the program continues to execute.

Hard Failure Program crashes (stack overrun….)

11

Classifying Failures Cascaded Failure

Program crashes and takes down other programs

Catastrophic Failure Program crashes and takes down the

operating system or the entire system; A total system failure.

12

Software Reliability Engineering (SRE)

Phase SRE Activities

Requirements Determine the functional profile of the systemDetermine and classify important failuresIdentify client reliability needsConduct trade-off studies.Set reliability objectives.

13

Software Reliability Engineering (SRE)

Phase SRE Activities

Design Design and evaluate systems to meet reliability goals.Allocate reliability targets to components

14

Software Reliability Engineering (SRE)

Phase SRE Activities

Implementation Measure and monitor the reliability during implementation.Manage fault introduction and propagation.

15

Software Reliability Engineering (SRE)

Phase SRE Activities

System and Field Testing

Measure reliability growth (using reliability growth models).Track testing progress against reliability growth.

16

Software Reliability Engineering (SRE)

Phase SRE Activities

Post-delivery & Maintenance

Monitor field reliability against reliability objectives.

17

Software Reliability Engineering (SRE) – Some Observations

Activities of SRE span the entire software development process But the activities themselves do not

constitute a development process. Activities themselves need to be

supported by Analytical tools – reliability models Evaluation methods – Markov Chains.

18

Software Reliability Engineering (SRE) – Some Observations

Design and implementation reliable systems is supported by design principles and algorithms For example, from the discipline of “Fault

tolerant systems”.

19

Key points Reliability depends heavily on

determining, and in practice, detecting failures.

Generally, four types of failures. Software Reliability Engineering

Span the entire software development process