Software Quality Assurance and Quality Control presentation by -Somashekhar Paple

Software Quality Assurance and Quality Control

presentation by

-Somashekhar Paple

03/05/20072

Introduction

Successful and smooth running of software product is very important. The success of a software product depends on its ability to satisfy client needs consistently.

It is the quality of the product that separates the great application from good ones.

03/05/20073

Agenda

• Definitions

• The Process of QA

• Phase Specific QA Activities

• Techniques of Quality Control

• Standards and Concepts related to Quality

• Cost of Quality

• Conclusion

• Discussions

03/05/20074

Definitions

QUALITY

• The totality of characteristics of an entity that bear on its ability to satisfy stated or implied needs. –PMI

• In simple terms - meeting expected standards in performance, reliability and consistency.

03/05/20075

Definitions cont’d

• SQA- The process of evaluating overall project performance on a regular basis to provide confidence that the project will satisfy the relevant quality standards.

• SQC- The process of monitoring specific project results to determine if they comply with relevant quality standards and identifying ways to eliminate causes of unsatisfactory performance.

03/05/20076

• Definitions





• Cost of Quality

• Conclusion

• Discussions

03/05/20077

The Process of QAQA involves set of tasks • Formulating the Quality Plan• Applying Software Engineering Techniques• Conducting Formal Technical Reviews• Applying Testing Strategies• Enforcing Process Adherence• Change Management• SQA Audits• Records and Reporting

03/05/20078

The Process of QA cont’d

• Formulating the Quality PlanIEEE Standards for SQA Plan

– It identifies the quality aspects of the software product to be developed.

– It is tracked as a live plan through-out the SDLC

• Applying Software Engineering Techniques– For requirement gathering – Project estimation using SLOC, FP technique

03/05/20079


• Conducting Formal Technical Reviews– Basically to assess the quality and design of the

prototype– Aims at error detection at the early stage

• Applying Testing Strategies– Involves

• Unit Testing• Integration Testing• System Level Testing

03/05/200710


• Enforcing Process AdherenceIt ensures that the project-related procedures and standards are followed– Process Adherence w.r.t. Product Evaluation

• Ensures standards set for the particular project are followed

• Compliance of the software product to the standards defined in the project plan is verified

– Process Adherence w.r.t Process Monitoring• Ensures the appropriate procedures are followed in

product development as specified in the project plan

03/05/200711


• Change ManagementIt deals with the changes occurring in the software product

features because of customer request or other reasons– Change Control

• Formalizing the request for change• Evaluating the nature of change

– Measuring the Impact of the Change• Use Software quality metrics• Enables monitoring of the activities and proposed changes

03/05/200712


• SQA Audits– Ensures the proper control is maintained over

the documents required during SDLC– Also ensures the status of the activity performed

is recorded in the status report

• Records and Reporting– Collection and Circulation of the information

regarding the SQA activities– The results of every task performed under SQA

are reported and compiled for future reference.

03/05/200713

• Definitions





• Cost of Quality

• Conclusion

• Discussions

03/05/200714

Analysis

Design

Development

Testing

Implementation

Phase Specific SQA Activities

03/05/200715

Phase Specific SQA Activities

• Analysis Phase– Ensures that the Processes, Procedures and

Standards followed are appropriate– Assures that the software product requirements

are complete and testable– And clear identification of requirements as

• Functional Requirements• Performance Requirements• Interface Requirements

03/05/200716

Analysis

Design

Development

Testing

Implementation

Helps in getting the right requirements

03/05/200717

Phase Specific SQA Activities cont’d

• Design PhaseSQA Ensures– Adherence to approved design standards– Software requirements gathered are allocated to

respective software components– Documents pertaining to Interface Design,

Conceptual Design Review and Physical Design Reviews are maintained

– Placing the approved design under the configuration management

03/05/200718

Analysis

Design

Development

Testing

Implementation

Ensures requirement reviews and configuration management

03/05/200719


• Development and Testing PhaseSQA Ensures– Readiness of the deliverable items for testing– Tests are conducted as per Test Plan– Testing is complete– Test reports are complete and correct – Software Product and Documentation are ready

for delivery

03/05/200720

Analysis

Design

Development

Testing

Implementation

Ensures proper testing and documentation

03/05/200721


• ImplementationSQA activities include– Audit of results of testing and design activities– Reviewing the schedule contained in the Project

Plan and status of all the deliverables– Monitor Configuration Management Activities– Report non-conformance – Verifying the corrective actions

03/05/200722

Analysis

Design

Development

Testing

Implementation Audit the test results and reporting

03/05/200723

• Definitions





• Cost of Quality

• Conclusion

• Discussions

03/05/200724

Techniques of Quality Control

Quality Control -

The operational techniques and activities that

are used to fulfill requirements for quality.

• Software Testing

• Walkthrough

• Inspection

03/05/200725

Software Testing• Quality control activity performed to detect errors in

a software program.• To find the difference between the expected and

observed behavior of the program.• Should have a comprehensive test plan• A test plan would include

• Details of testing tools and strategies• Reviews and Inspection Details• Details of staff performing testing• Product release criteria• Entry and exit criteria

03/05/200726

Software Testing cont’d • Types of testing

– Based on the approach• White box testing• Black box testing

– Based on the levels • Unit Testing • Integration Testing• Validation Testing• Security Testing• Stress Testing• Performance Testing• Alpha Testing• Beta Testing

03/05/200727

Software Testing cont’d

• Unit Testing– Performed on a module– Performed by the developer– Requires detailed knowledge of design/code

• Integration Testing– For determining the functionality of the combined

software modules– For detecting the errors associated with interfacing– More relevant for client/server and distributed systems

03/05/200728


• Validation Testing– To test the conformance to the requirements – Configuration review is performed to ensure that

all the elements in the software are properly developed and catalogued

• Security Testing– To test the protection mechanisms– To verify the security system for protection

against the unauthorized access (hacking).

03/05/200729


• Stress Testing– Also known as load testing– To verify the maximum stress a system can

withstand

• Performance Testing– To test the run time performance of the

application– Should meet the performance criteria defined in

the requirements documents

03/05/200730


• Functional Testing– Performed to check the functionality of the application– To check the accuracy of the output

• Alpha Testing– Testing of the completely operational software by a set of

internal users/external users at the developer's site

• Beta Testing– Testing of the completely operational software by a set of

users– Done prior to the final release of the software

03/05/200731

Walkthrough• A software review process• Developer guides the progress of the review• Two approaches

– Group Reviews

Typically Involves• Developer• Moderator• Reader • Recorder

– Individual peer-desk check• Performed by a single person

03/05/200732

Basic Steps• Overview—When the producer satisfies the entrance criteria, the inspection

is scheduled. The producer then conducts an overview. It acquaints the restof the inspection team with the product to be inspected.

• Preparation—The inspection team members study the product. The timespent in preparing is controlled based on the size of the product in KLOC.The members may use a checklist to focus on significant issues.

• Inspection meeting—The moderator supervises the inspection meeting.Some approaches use a reader other than the producer to actually conductthe inspection. The recorder makes a complete record of issues raised. Allmembers of the inspection team sign the report. Any team member mayproduce a minority report if there is a disagreement.

• Rework—The producer reviews the report and corrects the product.• Follow-up—The moderator reviews the report and the correction. If it

satisfies the exit criteria, the inspection is completed. If not, the moderatorcan either have the producer rework the product or re-inspection can bescheduled.

Inspection

03/05/200733

• Definitions





• Cost of Quality

• Conclusion

• Discussions

03/05/200734

Standards and Concepts related to Quality

• ISO Standards• TQM- Total Quality Management• Kaizen- Continuous Improvement process• CMM- Capability Maturity Model

03/05/200735

ISO StandardsISO (International Organization for Standardization)• The work of preparing international standards is done by ISO technicalcommittees.• ISO 9001 and ISO 9004 are a set of complementary standards with a

focus on quality.• ISO 9001 specifies requirements for a quality management system thatcan be used for internal application, ISO certification, or for contractualpurposes.• ISO 9004 provides guidance on a wider range of objectives of a qualitymanagement system than ISO 9001. It emphasizes the continualimprovement of an organization’s overall performance, efficiency, andeffectiveness. Used in organizations whose top management wishes tomove beyond the requirements of ISO 9001 in pursuit of continualimprovement.

03/05/200736

TQM – Total Quality Management• ISO Definition:"TQM is a management approach for an organization,

centered on quality, based on the participation of all its members and aiming at long-term success through customer satisfaction, and benefits to all members of the organization and to society."

It includes the following steps• Meeting Customer Requirements • Reducing Development Cycle Times • Just In Time/Demand Flow Manufacturing • Improvement Teams • Reducing Product and Service Costs • Improving Administrative Systems Training

03/05/200737

KaizenContinuous Improvement Process (CIP) (Kaizen- Japanese word)– A concept which recognizes that the world is constantly changing and anyprocess that is satisfactory today may well be unsatisfactory tomorrow.– Focuses on 11 principles: constancy of purpose, commitment to quality,customer focus and involvement, process orientation, continuous improvement,system-centered management, investment in knowledge, teamwork,conservation of human resources, total involvement, and perpetual commitment.– Rather than manage the output of the project, the focus is on managing the totalprocess and sub processes. The process is held constant only after it has beenproven capable of the work. Hence, the product naturally meets therequirements.– CIP steps:• Define and standardize processes (and sub processes).• Assess process performance.• Improve processes.• Measure progress.

03/05/200738

CMM• Developed by SEI - 'Software Engineering Institute' at Carnegie-

Mellon University • CMM - 'Capability Maturity Model Integration'. It's a model of 5

levels of process 'maturity' that determine effectiveness in delivering quality software.

• QA processes involved in the model are appropriate to any organization. Organizations can receive CMMI ratings by undergoing assessments by qualified auditors.

• Level 1- Initial Characterized by chaos, periodic panics, and heroic efforts required by individuals to successfully complete projects. Few if any processes in place; successes may not be repeatable.

• Level 2 – RepeatableSoftware project tracking, requirements management, realistic planning, and configuration management processes are in place; successful practices can be repeated.

03/05/200739

CMM cont’d

• Level 3 – DefinedStandard software development and maintenance processes are integrated throughout an organization; a Software Engineering Process Group is is in place to oversee software processes, and training programs are used to ensure understanding and compliance.

• Level 4 – ManagedMetrics are used to track productivity, processes, and products. Project performance is predictable, and quality is consistently high.

• Level 5 – OptimizedThe focus is on continuous process improvement. The impact of new processes and technologies can be predicted and effectively implemented when required.

03/05/200740

• Definitions





• Cost of Quality

• Conclusion

• Discussions

03/05/200741

• Quality costs are the costs associated with preventing, finding, and correcting defective work. These costs can be huge, running at 20% - 40% of the actual development work.

• Many of these costs can be significantly reduced or completely avoided.

• One of the key functions of a Quality Engineer is the reduction of the total cost of quality associated with a product.

Cost of Quality

03/05/200742

Different Types of Costs

• Prevention Costs: Costs of activities that are specifically designed to prevent poor quality.

• Appraisal Costs: Costs of activities designed to find quality problems, such as code inspections and any type of testing.

• Failure Costs: Costs that result from poor quality, such as the cost of fixing bugs and the cost of dealing with customer complaints. It can be of two types– Internal– External

03/05/200743

Cost Benefit Analysis• Model for Optimum Quality Costs, adopted from Mr. Juran’s Model

03/05/200744

Cost of Quality MatrixPrevention Staff training Requirements analysis Early prototyping Fault-tolerant design Defensive programming Usability analysis Clear specification Accurate internal documentation Evaluation of the reliability of developmenttools (before buying them) or of otherpotential components of the product

Appraisal Design review Code inspection Glass box testing Black box testing Training testers Beta testing Test automation Usability testing Pre-release out-of-box testing by customerservice staff

Internal FailureBug fixesRegression testingWasted in-house user timeWasted tester timeWasted writer timeWasted marketer timeWasted advertisementsDirect cost of late shipmentOpportunity cost of late shipment

External FailureTechnical support callsPreparation of support answer booksRefunds and replacement with updatedproductLost salesPR work to soften drafts of harsh reviewsLost customer goodwillWarranty costs and other costs imposed bylaw

03/05/200745

• Definitions





• Cost of Quality

• Conclusion

• Discussions

03/05/200746

ConclusionSoftware Quality Assurance and Quality

Control processes are very important for any organization to improve its processes. And these improvements will bring down the resources required for performing the a set of tasks over a period of time.

But the implementation these processes has to done in a phased manner over a period of time so as to protect the primary goal of the organization.

03/05/200747

Discussions…!

03/05/200748

IEEE Standards for SQA Plan

The following are the highlights of the sections specified in IEEE Std 730-1989:

1. Purpose—This section shall list the objectives of the software project.2. Reference Documents—This section shall list all the documents

referenced in the plan.3. Management

3.1 Organization—This section shall describe the structure of organization and the responsibilities, and usually includes an organizational chart.3.2 Tasks—This section shall list all of the tasks to be performed, therelationship between tasks and checkpoints, and the sequence of thetasks.3.3 Responsibilities—This section shall list the responsibilities of eachorganizational unit.

03/05/200749

IEEE Standards for SQA Plancont’d4. Documentation

4.1 Purpose—This section shall list all required documents and state howdocuments will be evaluated.4.2 Minimum documents—This section shall describe the minimumrequired documentation, usually including the following:SRS—Software Requirements SpecificationSDD—Software Design DescriptionSVVP—Software Verification and Validation PlanSVVR—Software Verification and Validation ReportUser documentation—Manual, guideSCMP—Software Configuration Management Plan

03/05/200750

IEEE Standards for SQA Plancont’d5. Standards, Practices, Conventions, and Metrics

This section shall identify the S, P, C, and M to be applied andhow compliance is to be monitored and assured. The minimalcontents should include documentation standards, logic structurestandards, coding standards, testing standards, selected SQA product,and process metrics.

6. Reviews and Audits—This section shall define what reviews/audits will be done, how they will be accomplished, and what further actions arerequired.

7. Tests—This section shall include all tests that are not included inSVVP.

8. Problem Reporting—This section shall define practices and procedures for reporting, tracking, and resolving problems, including organizationalresponsibilities.

03/05/200751

IEEE Standards for SQA Plancont’d9. Tools, Techniques, and Methodologies—This section shall identify the special

software tools, techniques, and methodologies and describe their use.10. Code Control—This section shall define the methods and facilities to maintain

controlled versions of the software.11. Media Control—This section shall define the methods and facilities to

identify, store, and protect the physical media.12. Supplier Control (for outsourcing)—This section shall state provisions for

assuring that software provided by suppliers meets standards.13. Records—This section shall identify documentation to be retained and

methods to collection, maintain, and safeguard the documentation.14. Training—This section shall identify necessary training activities.15. Risk Management—This section shall specify methods and procedures for

risk management.

03/05/200752

1. Purpose

2. References

3. Responsibilities

Task Responsibility Date

4. Documentation

Document Name Author Details

5. Standards and Guidelines

Document Details Phase

6. Audit Schedule

Serial No Planned Date Actual Date Nonconformities

7. Testing Strategy and Plan

-Objective

-Schedule

Activity Planned Start Date Planned End Date Actual Start Date Actual End Date

SQA Plan Template

03/05/200753

7. Testing Strategy and Plan

-Objective

-Schedule

Activity Planned Start Date Planned End DateActual Start Date

Actual End Date

8. Defect Reporting and Corrective Action

Defect Location CommentsDeveloper Comments Status

9. Tools Techniques and Methodologies

-Testing Tools

-Analysis and Desing Tools

-Configuration Management Tools

10.SCM

Baseline PhaseAcceptance Criteria

Approval authority Planned Date

Actual Date

11. Training Needs

Name Skills Required Training RequiredTraining Mode Planned Date

Actual Date

12. Risk Management

Risk DescriptionProbability of Occurrence Impact on project

Mitigation Steps End Date

Back

SQA Plan Template cont’d

03/05/200754

Click here to go back