96 Computer

Bin

ary

Cri

tic

Ikicked off this department in Feb-ruary by examining the gap betweenwhat we say and what we do in soft-ware projects (“The Hard RoadFrom Methods to Practice,” Feb.

1997, pp. 129-130). In any given project,our publicly declared methodologiesoften bear little resemblance to ouractual practices. In this article, I’d like toexamine one actual practice that is finallyemerging as a describable method: GoodEnough Software.

A few months ago, at a software devel-opment conference, I heard LarryConstantine declare that he doesn’tbelieve in the idea of good enough soft-ware quality. Constantine’s disdain wasso clear, I asked him why he even both-ered to dignify the idea with a critique.His reply: “Because everybody’s talkingabout it.”

I too hear talk about Good Enough,but it comes almost exclusively frompeople on software projects, rather thanfrom people who write or consult aboutsoftware processes. It’s hard to find muchof anything about Good Enough in soft-ware engineering literature, or at the con-ferences. There was an article in theCommunications of the ACM a fewyears ago (W. Robert Collins et al.,“How Good Is Good Enough?: AnEthical Analysis of Software Constru-ction and Use,” Jan. 1994, pp. 81-91).I’ve written one obscure article on thesubject (“The Challenge of Good

Enough Software,” American Program-mer, Oct. 1995) and spoken about it hereand there. Ed Yourdon has written aboutit, too, mostly in reference to my workand to his experiences interviewingdevelopers at Microsoft. Although thebasic ideas can be found in the work ofGerald Weinberg, Robert Glass, FredBrooks, and many others, I believe it wasYourdon who first elevated the notionfrom plain old English phrase to full-fledged technical buzzword.

A new buzzword! Like a new islandrising out of the Pacific, another long-neglected bit of software reality is burst-ing into the canon of software eng-ineering methodology. But as with anyvolcanic event, this new idea is subject tochaos and confusion.

CHAOS AND CONFUSIONFor example, Capers Jones devoted

five pages of his recent book (SoftwareQuality: Analysis and Guidelines forSuccess, International Thomson Press,1993) to debunking what he called the“good enough quality fallacy.” As Jonesput it, “from the observed fact that mostcommercial software contains bugs atdelivery, the ‘good enough’ enthusiastshave formed the hypothesis that leavingbugs is a deliberate and even clever strat-egy on the part of commercial softwarehouses, which might advantageously beimitated by other software developers.”He also stated, “Companies such as

Microsoft do not deliberately ship soft-ware that contains bugs.”

I believe this reveals a poor under-standing of what I consider to be theGood Enough approach. And how do Iknow what Good Enough Software is?Because Good Enough Software—as I define it—is just a refinement of what Iexperienced virtually every day for 12years as a developer and test manager inmajor commercial software companies.My model is not based on some armchairhypothesis—it describes my real experi-ence. Furthermore, my company, ST Labs,has run testing projects for hundreds ofcompanies, and we find that it is routinefor our clients to ship with known bugs.

Cem Kaner, in Testing ComputerSoftware (International Thomson Press,1993)—standard issue to Microsoft’stesters—also refers to bug fix deferral asa routine practice. I know this is true atMicrosoft because my company runs a

dedicated test lab there. I speak and teachthere on a regular basis. My brother iseven a test lead there. Take it from me,Microsoft begins every project with thecertain knowledge that they will chooseto ship with known bugs. This strategyworks for Microsoft because it knowshow to ship with the right bugs.

GOOD ENOUGH AS A PARADIGMConfusion about Good Enough

Software is understandable and forgive-able, since no one has published an actualdetailed description of what GoodEnough means. Jones seems to define itas the practice of deliberately leavingbugs in the code so as to shorten theschedule. I’ve heard other people define itas providing the minimum quality thatyou can get away with.

Good EnoughQuality: Beyond

the BuzzwordJames Bach, ST Labs Inc.

So

ftw

are

Re

ali

tie

s

Take it from me, Microsoftbegins every project with

the certain knowledge thatthey will choose to ship

with known bugs.

.

August 1997 97

One reason why Good Enough is notbetter described may be that it is so mucha part of our experience that it seems tooobvious to mention. Only when con-trasted with idealistic, normative mod-els of software engineering does GoodEnough stand out as a separate para-digm—a different pattern of assumptionsabout the way the world works.Methods for achieving Good Enoughquality are of interest mainly within the

larger context of the Good Enough par-adigm.

Here are some of the basic assump-tions that I believe are part of the GoodEnough paradigm:

• We are obliged to cope with a worldfull of complexity, unknowns, lim-itations, mistakes, and generalimperfection.

• People are by far the most variable

and vital components of softwareprojects.

• Everything has a cost, and what wewant always exceeds what we canafford.

• Quality is ultimately situational andsubjective.

• To achieve excellence in somethingas complex as software, we have tosolve a lot of difficult problems, makea lot of trade-offs, and resolve con-

A Framework for Good Enough QualityTaken together, these four GEQ factors and six GEQ perspec-

tives comprise a robust set of reminders that can help frame a con-versation or make a convincing case about shipping a product,improving it, or implementing some better practice. For instance,when someone tells me that “good enough is not good enough,”I remember the stakeholder and critical purpose perspectives andtranslate that apparently paradoxical statement into something Ican question, such as “good enough for you is not good enoughfor me” or “good enough to survive is not good enough to suc-ceed.” Then the dialogue becomes one of examining whose val-ues matter or what purpose we are really trying to achieve.

GEQ FactorsThis four-part process expands upon the GEQ definition. It’s

not a rigorous formula, by any means. These factors aredesigned to help busy, stressed-out software people rememberwhat to think about when weighing product quality.

1. Assess the benefits of the product.• Identification. What are the known benefits or potential

benefits for stakeholders of the product?• Likelihood. Assuming the product works as designed,

how likely are stakeholders to realize each benefit?• Impact. How desirable is each benefit to stakeholders?• Individual criticality. Which benefits, all by themselves,

are completely indispensable?• Overall benefit. Taken as a whole, and assuming no prob-

lems, are there sufficient benefits for stakeholders?

2. Assess the problems of the product.• Identification. What are the problems or potential prob-

lems for stakeholders of the product?• Likelihood. How likely are stakeholders to experience

each problem? • Impact. How damaging is each problem to stakeholders?

Are there workarounds?• Individual criticality. Which problems, all by themselves,

are completely unacceptable?• Overall problems. How do all the problems add up? Are

there too many noncritical problems?

3. Assess product quality.• Overall quality. With respect to the GEQ perspective, do

benefits appear to outweigh problems?• Margin of safety/excellence. By how much do we need

or want benefits to outweigh problems?

4. Assess the logistics of improving the product (or holding out forsomething better).

• Strategies. What strategies can we use to improve theproduct?

• Capabilities. How able are we to implement those strate-gies? Do we know how?

• Costs. How much cost or trouble will improvemententail? Is that the best use of resources?

• Schedule. Can we ship now and improve later? Can weachieve improvement in an acceptable time frame?

• Benefits. How specifically will it improve? Are there anyside benefits to improving it (for example, better morale)?

• Problems. How might improvement backfire (for exam-ple, introduce bugs, hurt morale, starve other projects)?

GEQ PerspectivesThe GEQ factors just listed are necessary but not sufficient.

In order to perform a responsible assessment, you must alsoexamine each of the factors from six vital perspectives.1. Stakeholders. Whose opinion about quality matters? (For

example, project team, customers, trade press, courts oflaw.)

2. Critical purpose. What do we have to achieve? (Is it imme-diate survival, profitability, market share, market growth,customer satisfaction?)

3. Time frame. What is the time-sensitivity of quality per-ception? (Is it immediate, near-term, long-term, after crit-ical events?)

4. Alternatives. How does this product compare to alter-natives, such as competing products, services, or solu-tions?

5. Consequences of failure. What if quality is a bit worsethan good enough? Do we need a contingency plan?

6. Quality of assessment. How confident are we in ourassessment? Is it good enough?

.

98 Computer

tradictory values. Excellence does notcome easily or mechanically.

• Software engineering methods are use-ful to the extent that they are designedwith these assumptions in mind.

Bear in mind that the real essence ofGood Enough lies in the minds of practi-tioners, not in any practice. The paradigmis one of learning on the job, learning fromfailure, coping with complexity, and cop-ing with humanity. It encourages healthyskepticism by building in the idea thatbenefits always come with problems. Ourtask is not to blindly eliminate all prob-lems, but to understand the problems andbenefits of a situation well enough to elim-inate (or prevent) the right problems andalso deliver the right benefits.

As a consultant, I see the GoodEnough approach mostly as a way todrive ongoing improvement, wherebywe approach excellence by progressivelyachieving, challenging, and raising ourstandard of Good Enough, as opposedto driving toward some abstract, apoc-ryphal metric like six sigma, or a defectremoval ratio of 99 percent, or a CMMlevel of 4. It’s also useful as a Rosettastone that helps quality specialists dis-cuss quality with people in other spe-cialties.

GOOD ENOUGH QUALITYANALYSIS FRAMEWORK

There are a few of us in the industrywho are quietly toiling away to developand teach this as a discipline. Here I pro-pose a framework for evaluating GoodEnough Quality (GEQ). Let’s start witha definition of what’s GEQ. Here’s mywhack at it. To claim that any given thingis Good Enough is to agree with all of thefollowing propositions:

• It has sufficient benefits.• It has no critical problems.• The benefits sufficiently outweigh

the problems.• In the present situation, and all

things considered, further improve-ment would be more harmful thanhelpful.

Each point is critical. If any one of themis not satisfied, then the product, although

perhaps good, cannot be good enough.The first two seem fairly obvious, but

note that they are not exact opposites.The complete absence of problems can-not guarantee infinite benefits, nor caninfinite benefits guarantee the absence ofproblems. Benefits and problems do off-set each other, but it’s important to con-sider the product from both perspectives.

The third proposition reminds us thatbenefits must not merely outweigh prob-lems, they must do so to a sufficientdegree. It also reminds us that even in theabsence of any individual critical prob-lem, there may be a pattern of noncriti-cal problems that essentially negate thebenefits of the product.

The fourth proposition introduces theimportant matter of logistics and sideeffects. If high quality is too expensive toachieve, or if achieving it would causeother unacceptable problems, then weeither must accept lower quality as beinggood enough or we must accept that a good enough product is beyond ourreach.

These propositions can be expandedinto a framework of GEQ factors thatsupport a process of structured reason-ing and dialog about quality, as summa-rized in the sidebar “A Framework forGood Enough Quality.”

RATIONAL, NOT COMPULSIVEI hope the framework makes it clear

that Good Enough has nothing to dowith mediocrity. It has to do with ratio-nal choices, as opposed to compulsivebehavior. If something really is goodenough, in terms of this framework, thenfurther improvement means making aninvestment that has an inadequatereturn. When we find ourselves in thatsituation, we should look for hiddencompulsions.

Is there anything new with the GoodEnough approach? Not really. Maybejust its name. Henry Petroski has beenwriting about the role of failure in suc-cessful design for years. Herbert Simoncoined the term “satisficing” in hisbook, The Sciences of the Artificial(MIT Press, 1992). Two excellent bookson Good Enough software engineering(although neither book bills itself assuch) are Gerald Weinberg’s RethinkingSystems Analysis and Design (DorsetHouse, 1988), and Robert Glass’ Soft-ware Creativity (Prentice Hall, 1995). Afew years ago, I couldn’t find any booksabout software risk management; nowthere are more than a dozen in print.

The big new force that is propellingthe Good Enough idea is the explo-sion of market-driven software.

With a passion roughly proportional tothe price of Microsoft stock, companiesare looking for the shortest path to bet-ter software, faster, and cheaper. They arewilling to take risks, and they have littlepatience for the traditional moralisticarguments in favor of so-called goodpractices.

Much of the traditional lore of soft-ware project management seems irrele-vant or stilted when applied to market-driven projects.

It’s time that we developed approachesand methodologies that apply to thewhole craft, not just to space missions,medical devices, or academic experiments.Good Enough is a model that encom-passes high-reliability products as well ashigh-entertainment products. Whetheryou call the idea Good Enough, or chooseanother buzzword like economical, prag-matic, or utilitarian, the basic idea remainsthe same: Our behavior should be guidedby reason, not compulsion.

Beyond the notion of “best practices”is a more fundamental idea: best think-ing. As the Good Enough idea continuesto emerge, the quality of our thinking,rather than our conformance to formal-ities, will become the issue. Formalities,and the authority behind them, will bereexamined. No wonder so many au-thorities consider Good Enough to be adangerous idea. ❖

Good Enough has nothing to do with

mediocrity. It has to dowith rational choices, asopposed to compulsive

behavior.

.