CSE 8314 - SW Metrics and Quality Engineering Copyright © 1995-2001, Dennis J. Frailey, All Rights Reserved CSE8314M01 8/20/2001Slide 1 SMU CSE 8314

8/20/2001

Slide 1CSE 8314 - SW Metrics and Quality EngineeringCopyright © 1995-2001, Dennis J. Frailey, All Rights Reserved CSE8314M01

SMU CSE 8314 / NTU SE 762-N

Software Metrics and Quality Engineering

Module 01

Overview of Software Quality Engineering

8/20/2001


Why is there So Much Ineffective Product

Development?

• Organizations focus on cost or schedule ...

… instead of looking at the big picture

8/20/2001


The “Zero-sum Game” Trap

PickAnyTwo

Quality

ProductivityCyc

le T

ime

8/20/2001


The Secret to Effective Product Development

• Make the Process Efficient– Eliminate waste– Eliminate mistakes– This makes things faster, less costly, and

higher in quality

Avoid the mistake of seeing the problem as a zero sum game , such as:

“to cut cost or save time you must reduce quality”; “to improve quality you must make the

product more expensive.”

8/20/2001


Effective Quality Engineering is Fundamental to Productivity and Cycle Time Improvement

EffectiveProduct

Development

Quality

ProductivityCyc

le T

ime

8/20/2001


Any Banner Will Do

Total Quality Management

Total Cycle Time

Productivity Enhancemen

t

Six Sigma

8/20/2001


Defining Quality

8/20/2001


Concepts of Quality

Webster defines quality as: 1) “that which makes something what it is"2) “the degree of excellence”

But is this what we mean for software?1) “our software is what it is - that makes it

a quality product"2) “the more perfect the software the

higher the quality”

8/20/2001


“That Which Makes it What it Is”

• e.g. Purity of tone is a quality of music– But perhaps not in certain musical styles– What defines the quality of “hard rock” music?

Is quality in the ear of the beholder?Is there a universally accepted

characteristic of musical quality?

8/20/2001


“Degree of Excellence”• Which has higher quality: a Ferrari or a Toyota Corolla ???

• Which has more prestige?• Which costs less and leaves money for

other expenses?• Which is more reliable?• Which weighs more?

8/20/2001


Concepts of Quality for Products

“Quality is conformance to requirements”Crosby

“Quality is fitness for intended use”Juran

“Quality is value to someone”Weinberg

8/20/2001


“Quality isConformance to Requirements”

• If testable requirements can be established, then it is possible to decide whether the product meets the criteria

• Thus you can avoid disputes and have workable contractual relationships

HOWEVER ...

8/20/2001


Issues with “Conformance to Requirements” - I

• Who establishes the requirements?– Sponsor - The one who pays for the product– End User - The one who will use the product– Sales or Marketing - The one who will sell

the product– Engineering - The ones who will design and

build it

8/20/2001


Issues with “Conformance to Requirements” - II

• Are the requirements right?– consistent– complete– correct

• Who determines whether the requirements are right?

• What if you discover a problem later on?

8/20/2001


Issues with “Conformance to Requirements” - III

• What about implicit vs. explicit requirements?– E.g. coffee should be hot and flavorful– Implicit requirement: not poisonous

• Furthermore, requirements change during the development process– Who makes and who controls the change?– Who pays for the consequences of change?

8/20/2001


“Quality is Fitness for Intended Use”

• This definition is based on a fundamental concept of law - that a product should be fit for the use that it is intended for

• This definition accommodates the fact that we may not be able to fully define the requirements

HOWEVER ...

8/20/2001


Issues with “Fitness for Intended Use” - I

• Who defines fitness?– Consider a TV set -- which fitness

characteristics are not understood by• Typical User• Engineer• Sales Personnel

– Consider a software program -- which fitness characteristics are not understood by the typical software developer?

8/20/2001


Issues with “Fitness for Intended Use” - II

• Different users have different definitions of fitness– Ease of use for novices vs control of fine

details for experts– vs ease of maintenance for support staff

• Uses change as users grow in experience– Too many “ease of use” and “automatic”

features may frustrate an expert

8/20/2001


Issues with “Fitness for Intended Use” - III

• The “pleasant surprise” concept– User gets more than he or she expected– “They really knew what they were doing”

There is always a balance between the engineer knowing better than the

customer and the customer knowing better than the engineer

8/20/2001


“Quality isValue to Someone”

• This definition incorporates the idea that quality is relative

• And it places increased emphasis on understanding what quality means to the intended user of the software

HOWEVER ...

8/20/2001


Issues with “Value to Someone”

• Whose opinion counts?– May need to weigh different opinions– May need to separate explicit from implicit

views

• Logic vs Emotion– “Glitz” v. “Substance”

• What is it Worth?– Space Shuttle -- 0 defects– Video Game -- good user interface

8/20/2001


Definitions of Software Quality

• IEEE: The degree to which the software possesses a desired combination of attributes

• Crosby: The degree to which a customer perceives that software meets composite expectations

Note that both definitions imply multiple expectations

8/20/2001


Software Quality Characteristics

Defi nition End Marketing SW SW CS

User Dept Manager DevelopersProf s

User Friendly X

Compatible with Competitor X

Many Features X X X

High Perf ormance X X

0 Def ects X X

Rapid Development X X X

Low Development Cost X

Elegant Code X X

8/20/2001


Summary of Quality Definition Issues

• Define quality– You must define it to know if you have it– … and to engineer it into your product

• Quality has multiple elements– It reflects a multitude of expectations

• Quality is relative– Quality is in the eye of the customer

• Quality encompasses fitness, value, and other attributes

8/20/2001


Quality Engineering

8/20/2001


The Evolution of Quality

Quality Engineering

Quality Control

Quality Assurance

1916 today future1950’s

8/20/2001


Quality ControlPreventing unacceptable products from

being released to the customer

• Emphasis is on finding defects and fixing them after the fact.

“A regulatory process through which we measure actual quality performance, compare with standards, and act on

differences.”Juran

8/20/2001


Quality ControlGoal: Keep Quality at an Acceptable Level by

Rejecting Unacceptable Products

Requirements

Development QC InspectionPass

Fail

Standards ofQuality

8/20/2001


Headrest Story - Part I:Independence

Why go to college?

I’ll get a job at an automobile

assembly plant!

My Brother

8/20/2001


Headrest Story - Part II:Employment

I found a quality control job on the assembly

line ... finding defective headrests.

8/20/2001


RejectsMy Brother

Headrest Story - Part III:Excitement

The highlight of my day!!!

They switched from red to blue!

8/20/2001


Headrest Story - Part IV: Quality Control

QC Manager

Production

Manager

Production rate is too

low! You’re too

picky!

These are substandard! Pay more attention to the criteria!

8/20/2001


Headrest Story - Part V:“Discussion”

You’rea

%#*@*#&$%

You!

Discussion (as used in automobile assembly lines):

Verbal communication characterized by extensive use of profanity and threats of

bodily harm.

8/20/2001


Headrest Story - Part VI:The Following Fall

My Brother

8/20/2001


Problems with Quality Control

• Does not reduce the number of defects

• Does not improve the process

• Does not result in better products

• Does not motivate improvement

• Results in adversarial relationships

8/20/2001


Quality Assurance

Assuring Product Quality: “Building Quality In”

• Providing evidence that the quality function is being performed adequately

Juran• Quality assessment and measurement

Fisher/Baker

8/20/2001


Quality Assurance

“A planned and systematic pattern of all actions necessary to provide adequate confidence that the product conforms to established technical requirements”

IEEE (George Tice)

8/20/2001


Software Quality Assurance

Don Riefer

• These methods and procedures include:– Planning, measuring and monitoring of all

work performed by software engineers, software testers, etc.

A system of methods and procedures used to assure that the

software product meets its requirements.

8/20/2001


Quality Assurance Looks at the Entire Process

Requirements

Development QC InspectionPass

Fail

Standards ofQuality

Process and Design Standards

8/20/2001


Quality Assurance is More Effective than Quality Control . .

.... because the emphasis moves to

the development process

• You attempt to fix problems before and during the development process

• You improve the process and therefore reduce the number of defects in a lasting manner

8/20/2001


But ...• Quality improvement is still separate from

other process improvement and software development activities

• Adversarial relationships are still there– quality assurance vs. software developers– validating and testing vs. design and coding

• Motivation to improve is inconsistent• It costs more to have people monitoring

people

8/20/2001


Quality Engineering

• Similar to quality assurance, but the responsibility shifts to everyone on the team

• Quality is built into the development process– Requirements, Design, Coding, Testing, etc.

• This is a very professional and responsible approach to software development

8/20/2001


The Philosophical Change in QE

• Problems result in process changes, not punishment of people

• Finding errors is good -- it keeps them from leaking through to the customer

• Everyone appreciates that a competitive process is the way to remain a competitor

• Measurements are used so that decisions are based on fact (in addition to intuition)

8/20/2001


Quality Engineering Requires a Cultural Change

• Pride in quality in addition to pride in product features or performance

• Professionalism rather than fear of criticism

• Overcoming the fear of metrics• Seeing software development as much

more than programming and design

“We” rather than “They”

8/20/2001


Quality Engineering Approach

• Build quality into the product as part of the development process– Measure quality– Understand quality– Improve quality

• Engineer the whole process for improvements in quality, productivity and cycle time (“Process Engineering”)

• A defined process is a must !!

8/20/2001


Elements of Quality Engineering

• Understand process and its role• Define value and quality - and focus on

adding both of these to the product• Manage process performance through

programs such as six-sigma or zero defects or statistical process control

• Analyze the cost of quality• Define and manage software reliability

8/20/2001


Benefits of the Quality Engineering Approach

• Less adversarial• Motivation and information to improve• Flexibility to change the process in

response to a problem– you understand the problem and its cause– you understand the consequences of a

change in the process

• Knowledge is the foundation of successful quality engineering

8/20/2001


Summary

• Product development is not a “zero sum game”

• Quality must be defined in terms of things that matter to customers

• Quality engineering focuses on the whole process and involves the whole project team

8/20/2001


References

• Crosby, Philip B. Quality is Free, New York, McGraw-Hill, 1979.

• Deming, W. Edwards, Out of the Crisis, MIT Press, 1986, ISBN: 0911379010

• Juran, J. M., Juran on Leadership for Quality: An Executive Handbook, The Free Press, 1989.

• Juran, J. M. and Frank M. Gryna, Quality Planning and Analysis, McGraw-Hill. 1980.

8/20/2001


References (continued)

• Schwartz, James B., 1994, The Hunters and the Hunted, Productivity Press, ISBN 1-56327-043-9

• Weinberg, Gerald M., 1992, Quality Software Management, Volume 1, Systems Thinking, Dorset House, New York, ISBN 0-932633-22-6.

8/20/2001


END OFMODULE 01

Documents

CSE 8314 - SW Metrics and Quality Engineering Copyright © 1995-2001, Dennis J. Frailey, All Rights Reserved CSE8314M01 8/20/2001Slide 1 SMU CSE 8314