Software Testing

Anuradha Bhatia

BASICS OF SOFTWARE TESTING CONTENTS

I. Software Quality, Definition of testing, Role of testing.

II. Failure, Error, fault, Defect, Bug Terminology

III. Objectives of Testing

IV. What is a Test Case?

V. When to start and Stop Testing of software.

VI. Skills for Software Tester.

VII. Quality Assurance, Quality Control, Verification and Validation, V Model

Software Testing

Anuradha Bhatia

I. Software Quality, Definition of testing, Role of testing 1. Software Quality

Software quality is the degree of conformance to explicit or implicit requirements

and expectations.

i. Explicit: clearly defined and documented

ii. Implicit: not clearly defined and documented but indirectly suggested

iii. Requirements: business/product/software requirements

iv. Expectations: mainly end-user expectations

2. Definition of Testing

(Question: Explain testing with software quality- 2 Marks each, 4 Marks)

Testing can be defined in simple words as “Performing Verification and Validation of

the Software Product” for its correctness and accuracy of working.

i. Software testing is as old as the hills in the history of digital computers.

ii. The testing of software is an important means of assessing the software to

determine its quality.

iii. Since testing typically consumes 40% to50% of development efforts, and

consumes more effort for systems that require higher levels of reliability, it is

a significant part of the software engineering.

iv. With the development of fourth generation languages (4GL), which speeds up

the implementation process, the proportion of time devoted to testing

increased.

v. As the amount of maintenance and upgrade of existing systems grow,

significant amount of testing will also be needed to verify systems after

changes are made.

3. Role of Testing

(Question: Explain the role of Testing – 6 Marks)

i. Software testing is to identify errors in order to reveal and spot it.

ii. The extent of software testing consists of implementation of that code in

different domain and also to look at the features of the code does the software

do what it is should be done and methods respect to the condition.

iii. It is proposed to begin testing from the first phase of the software

development.

iv. It saves time as well as cost. It is a continuous method, which is probably

nonstop but has to be stopped anywhere, for the need of time and resources.

Software Testing

Anuradha Bhatia

v. The basic need of the testing is to provide best quality product without taking

so much time and money.

vi. The test engineer has to pursue some technical way by which he/she can

review that all the points of necessity for testing have been covered or not.

In SDLC stage there are some most importance things as described below:

i. Recognition of Error and Faults

Testing step is one step which resolves the errors and faults in the software application.

These errors may be in unit level or in system level. After going through so many testing the application will be free of errors

that may be disturbing the application.

ii. Statistics to Shareholders and Status of Organization

Testing stage helps to know the condition of product and work standards. The stakeholders get better data through testing stage about utility value

too.

iii. Enhancement in Product Standards

Testing can help to know the real result and the probable result. It also helps to pick up the standards of the software. With proper testing an application can come out of bugs and build up ideal

software for the end-users.

iv. Technical Significance

Testing segment is significant for technical characteristics of any SDLC, as the software then completed with technically satisfied.

v. To Succeed of any Contentious Programmers

Ideal testing functions and tools aid to evolve up the product in business and keep programmers away from the other contestant.

Going through all stages of testing, the software application will be more bugs free, protected and technically sound.

vi. Free from any Risk

Whenever going to develop any software, testing is an essential part.

Software Testing

Anuradha Bhatia

When develop software without any testing then it may cause lots of risks to the end users.

To free everyone from any risk, it is essential that to go under all testing stages.

vii. Enhanced Standards

Appropriate tested application provides additional assurance of build up with best software.

Moreover, it refines standards of application as incessant and all types of testing stages have prepared a protected and harmless software application that could be worn by the end users.

viii. Confirmation and Corroboration

One of the major targets of testing stage in SDLC is for confirmation and corroboration.

Testing is greatly used in confirmation and corroboration method. Depending on the result we can compare among standards of several

software application.

ix. Credibility Evaluation

Testing stage also insist this important issue. If the software application has gone through all the testing types (like unit

testing, regression testing etc.), the application will surely be a reliable one.

So, testing evaluate credibility of software application. Testing provides the greatest analytical process to give equipped testing

on product ensuing in a credible product.

x. Demonstrate Accessibility and Feasibility

One of the most significant targets of testing is to demonstrate the product is both accessible and functional.

Accessibility testing is where the application is delivering to a select assembly of users and their functioning with the application is noticed.

All type of a user's communication with the application, like easiness of applies and whenever users are getting troubles, are preserved and examined.

xi. Avoid Fault Immigration

Software Testing

Anuradha Bhatia

In the first stage of SDLC, most of the faults have been found. If the faults can be noticed earlier, then these may be prohibited from immigrating to the following progress stage.

If the errors could be discover previously then the saving of software development cost will be vast.

xii. Commercial Significance

A full tested software application will have excellent business aspects. As all are like to work with reliable and trusted application in commercial.

II. Failure, Error, fault, Defect, Bug Terminology (Question: Define Bug and list the terms related to software failure: – definition -

1 mark, list – 1 mark, explanation – 2 marks = 4 marks)

Definition: A bug can be defined in simple term as any error or mistake that leads to the

failure of the product or software either due to the specification problem or due to

communication problem, regarding what is developed and what had to be developed.

1. Bug Terminology: Failure, Error, Fault The various terms related to software failure with respect to the area of

application are listed as Defect, Variance, Fault, Failure, Problem, Inconsistency,

Error, Feature, Incident, Bug, and Anomaly.

i. Problem, error, and bug are probably the most generic terms used.

ii. Anomaly, incident, and variance don’t sound quite so negative and infer

more unintended operation than an all-out failure.

iii. Fault, failure, and defect tend to imply a condition that’s really severe,

maybe even dangerous. It doesn’t sound right to call an incorrectly colored

icon a fault. These words also tend to imply blame: “It’s his fault that the

software failed.

iv. As all the words sound the same they are distinguished based on the

severity and the area in which the software failure has occurred.

v. When we run a program the error that we get during execution is termed

on the basis of runtime error, compile time error, computational error, and

assignment error.

vi. The error can be removed by debugging, if not resolved leads to a problem

and if the problem becomes large leads to software failure.

vii. A bug can be defined as the initiation of error or a problem due to which

fault, failure, incident or an anomaly occurs.

Software Testing

Anuradha Bhatia

viii. The program to find the factorial of a number given below lists few errors

problem and failure in a program.

For example

1. #include<stdio.h>

2. void main()

3. {

4. int i , fact, n;

5. printf(“enter the number “);

6. scanf(“%d”,&n);

7. for(i =1 ;i <=n;i++)

8. fact = fact * i;

9. printf (“the factorial of a number is ”%d”, fact);

10. }

As in line number 4 the fact is not initialized to 1, so it takes garbage value

and gives a wrong output, this is an example of a bug.

If fact is initialized to zero (fact = 0) than the output will be zero as anything

multiplied by zero will give the output as zero. This is a bug which can be

removed by initializing fact = 1 during initializing.

As the fact is declared as integer, for the number till 7! will work perfectly.

When the number entered is 8, the output is garbage value as the integer

limit is from – 32767 to +32768, so in declaration change the initialization

to long int fact;

III. Objective of Testing (Question: Explain the objective of Testing- 6 Marks)

1. Finding defects which may get created by the programmer while developing the

software.

2. Gaining confidence in and providing information about the level of quality.

3. To prevent defects.

4. To make sure that the end result meets the business and user requirements.

5. To ensure that it satisfies the BRS that is Business Requirement Specification and

SRS that is System Requirement Specifications.

6. To gain the confidence of the customers by providing them a quality product.

Software Testing

Anuradha Bhatia

IV. When to start and stop testing of software 1. When to Start Testing

(Question: Give the steps when should software testing start in project development- 6 Marks)

i. An early start to testing reduces the cost, time to rework and error free software that is delivered to the client.

ii. Software Development Life Cycle (SDLC) testing can be started from the Requirements Gathering phase and lasts till the deployment of the

software. iii. It also depends on the development model that is being used. For example

in Water fall model formal testing is conducted in the Testing phase, but in incremental model, testing is performed at the end of every

increment/iteration and at the end the whole application is tested. iv. Testing is done in different forms at every phase of SDLC like during

Requirement gathering phase, the analysis and verification of requirements are also considered testing.

v. Reviewing the design in the design phase with intent to improve the design is also considered as testing.

vi. Testing performed by a developer on completion of the code is also categorized as Unit type of testing.

2. When to Stop Testing

(Question: Give the steps when should software testing stop in project development- 6 Marks)

i. Unlike when to start testing it is difficult to determine when to stop testing,

as testing is a never ending process and no one can say that any software is 100% tested.

ii. Following are the aspects which should be considered to stop the testing:

Testing Deadlines.

Completion of test case execution. Completion of Functional and code coverage to a certain point.

Bug rate falls below a certain level and no high priority bugs are identified.

Management decision.

iii. Testing should be stopped when it meets the completion criteria.

iv. Completion criteria should be based on Risks. v. Testing should be stopped when :

Test cases completed with certain percentage passed and test coverage is achieved.

Software Testing

Anuradha Bhatia

There are no known critical bugs

Coverage of code, functionality, or requirements reaches a specified point;

Bug rate falls below a certain level, now testers are not getting any priority 1, 2, or 3 bugs.

vi. The risk can be measured by Risk analysis but for small duration / low budget / low resources project, risk can be deduced by simply: -

Measuring Test Coverage.

Number of test cycles.

Number of high priority bugs.

V. Skills of a software tester (Question: List and explain skills of software tester- 8 Marks)

1. Analytical and logical thinking i. The major objective of testing is to identify the hidden errors, not simply

prove that the software works. ii. For a tester to be effective in his role, he must be able to analyze the given

business situation and judge all the possible scenarios. iii. He should have the capacity to identify and tackle unfamiliar problems and

should develop a strategy to validate it. iv. Creating situations and validating the application under test, before

presenting it to customers, can be done effectively only by a person who has strong analytical skills.

ii. A tester should be able to separate the whole into logical parts -- to examine a complex problem, its elements and their relationships.

iii. He should be able to develop a logical argument based on relationships between elements and propositions, as well as identify implications, relationships, redundancies and contradictions without leaving any room for inconsistency and ambiguity.

iv. He should be consistent in analyzing and solving complex, multi-step problems.

2. The ability to envision business situations i. A tester should be able to envisage real-time business situations through

mental mapping, abstracting the idea inferred from the specifications. ii. The real-time business scenarios should crystallize in a tester's mind, and

he should think about what the case is rather than what ought to be the case or what he believes the case is.

Software Testing

Anuradha Bhatia

iii. A tester should be able to anticipate complex problems, in addition to visualizing and articulating them.

iv. He should be able to do a complete system simulation rapidly and accurately.

v. In the present software development environments, it is hard to believe that teams/individuals will get enough time to do a series of conventional brainstorming sessions to finalize the concept mapping.

vi. Therefore, it is vital that a tester develop his conceptualization skills through mental mapping.

3. A sense of intellectual curiosity and creativity i. A tester should understand that being an intellectual and being

intellectually curious are not the same. ii. A tester should arguably be the latter one -- intellectually curious -- which

is all about asking questions and not about having answers. iii. He should believe in the pursuit of knowledge as a value in and of itself. iv. He should love asking questions and should not consider it a blow to his

ego if he is wrong about something. v. It is intellectual curiosity that motivates and prompts a tester to identify

interesting questions about the software being tested. vi. Thus, a tester should develop the skill to see what everyone else hasn't

seen, to think what no one else has thought of and to do what no one else has dared.

4. A "glocal" approach i. Software systems have become extremely complex.

ii. Most of the time, the system designed involves multiple stakeholders, and dealing with such systems is not always easy.

iii. A tester should be able to deal effectively with business situations marked by complexity and the number of interactions with third-party systems.

iv. He should be able to identify how the system under test interacts with other constituents of the system.

v. He should also be able to isolate the minutest units of the application under test and do the validation, keeping in mind the behavior of the system as a whole.

vi. A tester will stand out among his peers if he is able to detach himself from the system, look at it as an outsider, and present his findings on the relationship between the individual components convincingly in a logical fashion.

vii. At the same time, he should keep his eyes and ears open so as not to ignore issues that may be inherent in individual components.

viii. Thus, a "glocal" (global + local) approach is essential.

5. Critical thought and rational enquiry i. The quality of life of an individual and the quality of what he

produces/delivers depends largely on the quality of his thought process.

Software Testing

Anuradha Bhatia

ii. The thought process of a tester should be undistorted, impartial and without any prejudices.

iii. A tester should be able to take charge of the inherent structures and impose intellectual standards upon the software under test.

iv. He should be able to raise vital questions precisely and clearly, gather and assess relevant information, interpret it effectively in order to come to well-reasoned conclusions and solutions, and test those conclusions against the given criteria and standards.

v. A tester should be open minded, suspend any judgment in the absence of sufficient evidence to support a decision, and always abide by logical reasoning.

6. The ability to apply basic and fundamental knowledge i. Knowledge in the context of testing can be attributed as the fluid mix of

experience, values, contextual information and expert insight. ii. Those things provide a framework for evaluating the system under test.

One can attain knowledge by so many means, but that knowledge is worthwhile only when it adds value to situations encountered.

iii. A smart tester should be able to apply the knowledge attained over years of experience with the domain, process, product, customers, mistakes and successes in his testing.

iv. He should be able to make use of fundamental communication, mathematical and software application skills.

v. He should also be able to effectively apply the skills he has attained to practical situations.

7. Continue to learn i. Organizations and business environments change rapidly, which means

the approaches and processes that work well today will be outdated tomorrow.

ii. Therefore, it is imperative that a tester place priority on noticing, adapting and learning from change that is happening around him.

iii. That doesn't mean a tester should continually undergo training or certification, rather he should be open to learning from everything in life that comes he across.

iv. If he has gained basic and fundamental knowledge, then the rest can be achieved through self-directed learning.

v. Learning should be a lifelong habit.

8. Respect for truth and intellectual integrity i. A tester should be able to examine the piece of software under test and

the resulting processes, with focus on the given specification, and understand the behavior of the software.

Software Testing

Anuradha Bhatia

ii. Being human, a tester may have severe biases, prejudices and intolerances that prevent him from performing well.

iii. He should possess the intellectual integrity to correct those barriers in order to efficiently understand the nature of the software under test.

iv. He should also be willing to shrug off the set of practices and character traits that undermine his intellectual integrity.

v. And he should be able to exemplify intellectual virtues such as honesty, impartiality and openness to the views of others.

9. Planning, time management skills i. Planning is nothing but writing the story of the future.

ii. A tester needs to have a thorough plan and must develop a well-thought test strategy/approach.

iii. And that plan must be in place before work begins on any software testing assignment.

iv. It should describe the items and features to be tested, the test strategy and levels of testing, pass/fail criteria, suspension/resumption criteria, schedule, etc.

v. The plan developed should be monitored continually, and validations should be done through organized system feedback.

vi. Sticking to the plan and monitoring the progress in order to ensure timely delivery is key to any software testing assignment's success.

10. Effective communication skills i. A tester must be able to communicate his thoughts and ideas effectively,

using a variety of tools and media. ii. He needs to develop and use this skill throughout his career and should

learn to communicate effectively to the stakeholders so as to avoid ambiguities and inconsistencies.

iii. For example, printed presentations should be concise and to the point and should follow logically.

iv. The language should be pragmatic rather than philosophical, and arguments should be supported by facts.

v. In the case of oral presentations, the voice, body language and appearance of the presenter are as important as the content and visual aids.

vi. A tester should develop his skills to overcome shyness and any fear of speaking. He should also have good listening skills.

Software Testing

Anuradha Bhatia

VIII. Quality Assurance, Quality Control, Verification and Validation, V Model

1. Quality Assurance (Question: Explain the concept of Quality Assurance and Quality control in software

testing. - 2 marks + 2 marks =4 marks)

i. Quality Assurance: A part of quality management focused on providing

confidence that quality requirements will be fulfilled.

ii. All the planned and systematic activities implemented within the quality

system that can be demonstrated to provide confidence that a product or

service will fulfill requirements for quality

iii. Quality Assurance is fundamentally focused on planning and documenting

those processes to assure quality including things such as quality plans and

inspection and test plans.

iv. Quality Assurance is a system for evaluating performance, service, of the

quality of a product against a system, standard or specified requirement

for customers.

v. Quality Assurance is a complete system to assure the quality of products

or services. It is not only a process, but a complete system including also

control. It is a way of management.

2. Quality Control i. A part of quality management focused on fulfilling quality requirements.

ii. The operational techniques and activities used to fulfill requirements for

quality.

iii. Quality Control on the other hand is the physical verification that the product

conforms to these planned arrangements by inspection, measurement etc.

iv. Quality Control is the process involved within the system to ensure job

management, competence and performance during the manufacturing of the

product or service to ensure it meets the quality plan as designed.

v. Quality Control just measures and determines the quality level of products or

services.

3. Precision and Accuracy (Question: Explain with diagram the difference between precision and accuracy. :-

diagram – 1 mark, explanation – 3 marks = 4 marks)

Software Testing

Anuradha Bhatia

i. The Accuracy of a measurement system is the degree of closeness of measurements of a quantity to that quantity's actual (true) value.

ii. The Precision of a measurement system, also called reproducibility or repeatability, is the degree to which repeated measurements under unchanged conditions show the same results.

iii. Although the two words precision and accuracy can be synonymous in

colloquial use, they are deliberately contrasted in the context of the scientific method.

Diagram for Accuracy and Precision

Figure 1: Accuracy and Precision

For example –

1. A measurement system can be accurate but not precise, precise but not accurate, neither, or both.

2. If an experiment contains a systematic error, then increasing the sample inputs generally increases precision but does not improve accuracy.

3. The result would be a consistent yet inaccurate string of results from the flawed software.

4. Eliminating the systematic error improves accuracy but does not change precision.

4. Verification and Validation (Question: Explain the concept of verification and validation in software

testing. - 2 marks + 2 marks = 4 marks)

Verification

i. It makes sure that the product is designed to deliver all functionality to the

customer.

Software Testing

Anuradha Bhatia

ii. Verification is done at the starting of the development process. It includes

reviews and meetings, walkthroughs, inspection, etc. to evaluate documents,

plans, code, requirements and specifications.

iii. It answers the questions like: Am I building the product right?

iv. Am I accessing the data right (in the right place; in the right way).

v. It is a Low level activity

vi. Performed during development on key art facts, like walkthroughs, reviews

and inspections, mentor feedback, training, checklists and standards.

vii. Demonstration of consistency, completeness, and correctness of the software

at each stage and between each stage of the development life cycle.

Validation

1. Determining if the system complies with the requirements and performs

functions for which it is intended and meets the organization’s goals and user

needs.

2. Validation is done at the end of the development process and takes place after

verifications are completed.

3. It answers the question like: Am I building the right product?

4. Am I accessing the right data (in terms of the data required to satisfy the

requirement).

5. It is a High level activity.

6. Performed after a work product is produced against established criteria

ensuring that the product integrates correctly into the environment.

7. Determination of correctness of the final software product by a development

project with respect to the user needs and requirements.

5. V Model (Question: Draw the diagram of V Model – 4 Marks)

Software Testing

Anuradha Bhatia

Figure 2: The V Model