Advanced Software Engineering: Software Testing COMP 3702 (Lecture1) Sada Narayanappa Seif Azgandhi Anneliese Andrews Thomas Thelin Carina Andersson

Advanced Software Engineering: Software

TestingCOMP 3702 (Lecture1)

Sada NarayanappaSeif Azgandhi

Anneliese Andrews Thomas ThelinCarina Andersson

A Andrews - Software Engineering: Software Testing'06

27 dm3, 31 kg

4 KB read/write magnetic core memory

64 KB read-only memory



Software on Wings4 M LOC


Facts about testing

System development:1/3 planning1/6 coding1/4 component test1/4 system test[Brooks75]


Implementation

Analysis

Design Function test

System Test


Good enough quality

To claim that any given thing is good enough is to agree with all of the following propositions:

It has sufficient benefits

It has no critical problems

The benefits sufficiently outweigh the problems

In the present situation, and all things considered, further improvement would be more harmful than helpful

James Bach, IEEE Computer, 30(8):96-98, 1997.


Quality attributes – ISO 9126


Quality attributes – McCall


Why use testing?

Risk mitigationFaults are found earlyFaults can be preventedReduce lead-timeDeliverables can be reused…


Why do faults occur in software?Software is written by humans

Who know something, but not everything Who have skills, but aren’t perfect Who don’t usually use rigorous methods Who do make mistakes (errors)

Under increasing pressure to deliver to strict deadlines No time to check, assumptions may be wrong Systems may be incomplete

Software is complex, abstract and invisible Hard to understand Hard to see if it is complete or working correctly No one person can fully understand large systems Numerous external interfaces and dependencies


Origins of defects

Defect sources

Lack of educationPoor communicationOversightTranscriptionImmature process

Impact of software artifactsErrors

Faults / Defects

FailuresImpact from user’s

viewPoor quality software

User dissatisfaction

Fault model


Whoops, that’s my calculator


Testing, Verification & Validation

Definition 1Verification

is the product right?

Validation is it the right product?

Definition 2Verification

satisfies the conditions at the start of the phase

Validation satisfies the requirements

TestingThe process of evaluating a program or a

system


Definitions

Failure is an event, fault is a state of the software caused by an error

Error – human mistake Fault / Defect – anomaly in the software Failure – inability to perform its required functions Debugging / Fault localization – localizing, repairing, retesting.

A test case consists of: A set of inputs Execution conditions Expected outputs


Scripted and non-scripted testing

In scripted testing test cases are pre-documented in detailed, step-by-step descriptions

Different levels of scripting possible

Scripts can be manual or automated

Non-scripted testing is usually manual testing without detailed test case descriptions

Can be disciplined, planned, and well documented exploratory testing

or ad-hoc testing


Test oracle

An oracle is the principle or mechanism by which you recognize a problemTest oracle provides the expected result for a test, for example

Specification document Formula Computer program Person

In many cases it is very hard to find an oracle Even the customer and end user might not be able to

tell which is the correct behaviour


Principle 1 – purpose of testing

Testing is the process of exercising a software component using a selected set of test cases, with the intent of

1. Revealing defects

2. Evaluating quality


Principles

2: A good test case – When the test objective is to detect defects, then a good test case is one that has high probability of revealing a yet undetected defect(s)

3: Test result – The results should be inspected meticulously

4: Expected output – A test case must contain the expected output


Principles

5: Input – Test cases should be developed for both valid and invalid input conditions

6: Fault content estimation – The probability of the existence of additional defects in a software component is proportional to the number of defects already detected in that component

7: Test organization – Testing should be carried out by a group that is independent of the development group


Principles

8: Repeatable – Tests must be repeatable and reusable

9: Planned – Testing should be planned

10: Life cycle – Testing activities should be integrated into the software life cycle

11: Creative – Testing is a creative and challenging task


Test

Mat

urity

Mod

el


Goals of the course

Knowledge

Skills

Attitudes

A test specialist who is trained as an engineer should have knowledge of test-related principles, processes, measurements, standards, plans, tools, and methods, and should learn how to apply them to the testing tasks to be performed.


www.swebok.org


www.swebok.org


Lectures

Theory + discussions Cover the basic parts of software testing

1. Introduction2. Black-box testing, Reliability, Usability3. Inspections, white-box testing4. Lifecycle, documentation5. Organization, tools6. Metrics, TMM7. Guest lecture,

Technical

Overview

Technical / ManagerialManagerial

Economic


Lab sessions

Preparation, Execution, Report

1. Black-box testing

2. Usage-based testing and reliability

3. White-box testing

4. Inspection and estimation

5. Software process simulation


Project: Option 1

Learn a specific area of software testing Collect and summarize research information Critical thinking beyond the written

information Present information in a structured way Peer review


Examination

Written exam based on the book and lab sessionsLab sessions (approved)Project (F / G / VG)


Schedule

Read Course program Projects in Software Testing

Check homepageNot decided

Extra Lab dates


This week

Read course programLecture 2 tomorrow Project

Read Projects in Software Testing Exercise on Thursday Decide subject (deadline Thursday 17/3)

Lab Prepare lab 1

Read Burnstein 1-3 Prepare Burnstein 4,12


Project: Option 1

Research: solve a research problem; survey the state-of-the-art and identify the research problems in some area; develop and justify an extension to an existing technique; etc.Evaluation: apply and evaluate a technique or evaluate a commercial testing or, analysis tool.Practical: Use an existing technique to test a system or design and implement a prototype for a system.


Project: Option 1

Read Projects in Software Testing Divide in groups (2-3 persons)

Discuss with me

http://www.cs.du.edu/~andrews/andrews/COMP3702/FilesFromCD/Project/Project_SwTest.pdf


Project: Option 2

Experiment participation.Compare Fault finding

Unit test code inspection

Two sessions 3-4 less totalFull credit

Documents

Advanced Software Engineering: Software Testing COMP 3702 (Lecture1) Sada Narayanappa Seif Azgandhi Anneliese Andrews Thomas Thelin Carina Andersson