Software Quality Assurance

and Testing Fazal Rehman Shamil

Outline

Concepts, Issues Testing Purpose Testing Processes, Activities and Context Classification of Testing Techniques

Software Testing 2

Basic Concept

Software Testing 3

According to ANSI/IEEE 1059 standard,

• Testing can be defined as “A process of analyzing a software item to detect the differences between existing and required conditions (that is defects/errors/bugs) and to evaluate the features of the software item”.

State-of-the-Art

Software Testing 4

30-85 errors are made per 1000 lines of source code

Extensively tested software contains 0.5-3 errors per 1000 lines of source code

If testing is postponed, as a consequence: the later an error is discovered, the more it costs to fix it.

Error distribution: 60% design, 40% implementation. 66% of the design errors are not discovered until the software has become operational.

Lessons

Many errors are made in the early phases

These errors are discovered late

Repairing those errors is costly

It pays off to start testing real early

Software Testing 5

Software Testing 6

7

Who Tests the Software?

developer independent tester

Understands the system

but, will test "gently"and, is driven by "delivery"

Must learn about the system,but, will attempt to break it

and, is driven by quality

Software Testing

8

What is a “Good” Test? a high probability of finding an error not redundant. neither too simple nor too complex

"Bugs lies on corners and gathered at boundaries ..."

Boris Beizer

Software Testing

Basic Questions

What artifacts are tested? (code and some other artifacts) What to test?

from which view?(External or internal-Black/White box)

related: type of faults found? When to stop testing? - some coverage criteria or reliability

goal

Software Testing 9

10

Software TestingWhite-Box Testing

... our goal is to ensure that all statements and conditions have been executed at least once ...

Black-Box Testing

requirements

eventsinput

output

Software Testing

Functional vs. Structural Testing Key distinction-> Perspective on what

need to be checked/tested? Functional testing:

tests external functions. - as described by external specifications black-box in nature; -functional mapping: input => output

-without involving internal knowledge

Software Testing 11

Functional vs. Structural testing Structural testing:

tests internal implementations. =>components and structures.

white-box in nature; - “white" here = seeing through => internal elements visible.

really clear/glass/transparent box.

Software Testing 12

Black Box Testing

The simplest form of BBT is to start running the software and make observations in the hope that it is easy to distinguish between expected and unexpected behavior.

Another common form of BBT is the use of specification checklists, which list the external functions that are supposed to be present, as well as some information about the expected behavior or input-output pairing.

Software Testing 13

Black Box Testing

Focus of test execution during BBT is to observe the external behavior, to ensure orderly execution of all the test

cases to record execution information for

analysis and follow-up activities. If the observed behavior patterns cannot

be immediately identified as failures, information needs to be recorded for further analysis.

Software Testing 14

Black Box Testing

Once the execution result is obtained, either individually or as a set, analyses can be carried out to compare the specific behavior and output with the expected ones. This comparison to determine if it is expected behavior or if a failure occurred is called the testing oracle problem.

Software Testing 15

Black Box Testing

Examples:Compiler Example

An example of a black-box system would be a search engine. You enter text that you want to search for in the search bar, press “Search” and results are returned to you. In such a case, you do not know or see the specific process that is being employed to obtain your search results, you simply see that you provide an input – a search term – and you receive an output – your search resultsSoftware Testing 16

White Box Testing Structural testing verifies the correct implementation

of internal units, such as program statements, data structures, etc., and relations among them.

This is done through test execution by observing the program behavior related to these specific units.

The simplest form of WBT is statement coverage testing through the use of various debugging tools, or debuggers, which help us in tracing through program executions.

Software Testing 17

White Box Testing One of the advantages is that once a problem is

detected, it is also located. The problems of omission(forget) or design

problems cannot be easily detected through WBT, because only what is present in the code is tested.

The tester needs to be very familiar with the code under testing to trace through its executions. Consequently, WBT and related activities are typically performed by the programmers themselves because of their intimate knowledge of the specific program unit under testing. This dual role also makes defect fixing easy.

Software Testing 18

White Box Testing

Example: An example of a white-box system would be

in-circuit testing where someone is looking at the interconnections between each component and verifying that each internal connection is working properly.

Software Testing 19

Gray Box Testing Gray box testing is a software testing technique

that uses a combination of black box testing and white box testing.

Gray box testing is not black box testing, because the tester does know some of the internal workings of the software under test.

In gray box testing, the tester applies a limited number of test cases to the internal workings of the software under test. In the remaining part of the gray box testing, one takes a black box approach in applying inputs to the software under test and observing the outputs.

Software Testing2 21

This is particularly important when conducting integration testing between two modules of code written by two different developers, where only the interfaces are exposed for test.

An example of Gray Box Testing would be when the codes for two units/ modules are studied (White Box Testing method) for designing test cases and actual tests are conducted using the exposed interfaces (Black Box Testing method).

Gray Box Testing

Pros and Cons of Black Box Testing

Tester can be non-technical.

This testing is most likely to find those bugs as the user would find.

Test cases can be designed as soon as the functional specifications are complete.

Chances of having repetition of tests that are already done by programmer.

It is difficult to identify all possible inputs in limited testing time.

Pros and Cons of Black Box Testing

Software quality assurance (SQA) Software quality assurance (SQA) is a

process that ensures that developed software meets and complies with defined or standardized quality specifications. SQA is an ongoing process within the software development life cycle (SDLC) that routinely checks the developed software to ensure it meets desired quality measures.

Software Testing 24

Software Quality control

Quality control is the set of measures and procedures to follow in order to ensure that the quality of a product is maintained and improved against a set of benchmarks and that any errors encountered are either eliminated or reduced. The focus of quality control is to ensure that the product and product manufacturing are not only consistent but also in line with customer requirements.

Software Testing 25

Difference between SQA and SQC See image file in detail as provide you in

class.

Software Testing 26

Verification and validation

Verification: To check that we are building the product

right

Validation: To check that we built the right product.

Software Testing 27

Thank You!

Software Testing 28