AUTHOR: William C. BaerTITLE: General Plan Evaluation Criteria: An Approach to Making Better Plans

SOURCE: Journal of the American Planning Association v63 p329-44 Summer '97

The magazine publisher is the copyright holder of this article and it is reproduced with permission. Further reproduction of this article in violation of the copyright is prohibited.

How would you know a good plan if you saw one? The planners' answer is dangerously near to the apocryphal answer to a similar question about good art: "I don't know much about art [plans], but I

know what I like." As a profession, we have developed few guides.

Plan quality is difficult to define. Planners can often differentiate high quality plans from low quality ones, but they are hard pressed to explicitly define the key characteristics of plan quality. The planning literature is surprisingly narrow when it comes to what constitutes a good plan. The planning profession has generally avoided this normative question and focused instead on the methods and processes of plan making. (Berke and French 1994, 237-8)

A similar point about the appropriate criteria for determining "good plans" was made recently by Dalton and Burby (1994, 447) and implicitly echoed by Stiftel and Boswell (1994). As Alexander and Faludi (1989, 127) noted in the context of planning theory: "If planning is to have any credibility as a discipline or a profession, evaluation criteria must enable a real judgment of planning effectiveness: good planning must be distinguishable from bad." A further instance of professional remiss is the problem of terminology. In the scant literature on "plan evaluation" the profession uses terms like "plan assessment," "planappraisal," "plan testing," and "plan evaluation" virtually interchangeably. Making these distinctions and formulating appropriate criteria can demonstrably inform our understanding of plan making--and of making plans better.

Despite intellectual neglect, the plan remains one of the planner's primary tools to influence future growth and development (Dalton 1989). Moreover, societal interest in the plan has intensified. At least fourteen states now have comprehensive growth programs, either statewide or aimed at particular subareas (Bollens 1992; Morgan 1993). State governments are increasingly mandating thorough, methodical, even exhaustive requirements for local plan preparation (Gale 1992; Innes 1992; Buschbaum and Smith 1993). Local citizen groups and developers alike, although often opposed over plan substance, nevertheless participate in its formulation and carefully monitor its implementation.

This faith placed in the plan is gratifying. We must justify it by better evaluating plans to carry the weight of responsibility the public has given to them. The evaluation should be carried out with close attention to its criteria. The first section of this article, therefore, examines the various meanings given to plan evaluation, and distinguishes the several stages during the plan-making process where evaluation can appear. Next, this section briefly reviews past efforts at plan conceptualization in order to articulate the criteria they imply. Appropriate criteria for evaluating plans are imbedded in the view of the plan to begin with; explicitly stating them is a way of understanding the plan's initial conceptualization.

The second section here presents another form of evaluation--assessing the plan as embodied in the document. I argue that planners must devise criteria for this stage in order to fully understand what they are doing while preparing the plan. A composite table of pertinent criteria is presented to illustrate the concept of assessment. They are not, however, meant as an approved recipe that, mechanically followed, will assure professional competence. Rather, they are intended to illustrate why and how individual professional planners must develop such criteria themselves each time they work on a plan.

In this regard, the criteria reflect a concern about plan adequacy and competence similar to that shown in state mandates for plans. State mandates are exhaustively--even mind-numbingly--"rational" in their detailed requirements for technical competence. They enhance this technical rationality by introducing criteria for intergovernmental consistency, administrative responsibilities, and agency time schedules. In this "administrative regulatory model" (Brooks 1979),

the plan becomes an interlocking directorate of state requirements and criteria that planners must contend with.

Most of the profession, however, would agree with Krieger's (1981) elegant paean to planning and plans as being more than conventional norms of expertise, or state lists of mandated planning tasks. Postmodern theorists go even farther, questioning the very premises of the state mandates, or pointing out that a plan is as much symbol as instrument, as much rhetoric as technical substance, and that the "public interest" amounts simply to irreconciable discourses to multiple publics. It follows, in this view, that plans should not aspire to such traditional (modernist) evaluation criteria as are presented here or in the state mandates.

Is it possible for postmodern views to reside comfortably in legislative mandates? How should planners deal with this conflict? The article's conclusion, in looking at this dilemma, suggests the problems inherent in operationalizing postmodern views. Evaluation criteria will remain useful to planners and help them respond to state mandates, so long as planners do not become totally enamored with technique and expertise.

PLAN EVALUATION CRITIERIA

DIFFERENT STAGES FOR PLAN EVALUATION

Appropriate criteria for plan evaluation depend on distinguishing the different stages in plan-making when evaluation can be performed. The rubric of evaluation has included: (1) plan assessment, (2) plantesting and evaluation, (3) plan critique, (4) comparative research and professional evaluations, and (5) post hoc evaluation of plan outcomes. The same word sometimes has been used for all those different phases. To distinguish them, it is helpful to identify who is undertaking the evaluation, and their relation to the plan authors; when the evaluation is undertaken, (i.e. at what stage during plan preparation or after its completion); and, finally, the what of the evaluation. The "what" takes several forms:

1. the substance of plan alternatives; and/or2. the plan as a package--including the document that communicates:

i. goals and objectivesii. needs or problems

iii. assumptions and method of reasoningiv. specific proposalsv. perhaps, implementation devices (ordinances, budgets, etc.) and/or

3. the outcome following plan implementation.

Several types of plan evaluations can be performed, and at several stages in the plan-making process. Figure 1 illustrates the various types and stages. (Figure 2 will show the different types of post hoc planevaluation.)

Plan Critique. Historically, the first type of plan evaluation was a critique, which is an evaluation similar to a book or movie review. The critique is by persons other than the plan's authors, but nevertheless typically trained as professional planners. The critique is undertaken after the plan's publication, but usually before it has had time to be put into practice, and certainly before any measurable results have occurred. The criteria being invoked by the critic are individual, implicit, and somewhat idiosyncratic, being based on the critic's professional virtualities--his or her art of judgment and conceptive powers (Jamous and Peloille 1970). The topics covered also depend upon the critic, but include the three content areas listed above, and usually emphasize the second one—the plan as document that communicates substance. Examples are found in the Plan Reviews presented periodically in the old Journal of the American Institute of Planners.(FN1) A careful reading of these reviews discovers the need for more explicitcriteria; they are lively and substantively informative, but rarely systematic, much less analytic. Still, the critic's approach has some virtues. Its very

individuality has the flexibility to form the critique to fit the occasion, and to introduce new concerns, such as those expressed by postmodernists (e.g., Healy's (1993) postmodern criteria, discussed later).

Plan Testing and Evaluation. The term, plan evaluation, gained prominence in the early 1970s as a term for the process of evaluating alternative ways to achieve a plan's goal (Robinson 1972; Lichfield, Kettle, and Whitebread 1975). Planners were to adopt the best of the alternatives after analyzing and testing them. Lichfield, Kettle, and Whitebread's 11-part planning model suggested the following vocabulary and steps for evaluation in the plan-making process: testing the alternative plans (Phase 7); evaluating these tests (Phase 8); choosing the best (phase 9) (in figure 1 these phases are condensed into steps 5 and 6 for simplicity); implementing the plan (Phase 10); and, in Phase 11, comparing the outcomes with the predicted outcomes, and appraising the significance of any unanticipated consequences (Lichfield, Kettle, and White-bread 1975, 19-22).

Note that several new aspects were introduced. The evaluation is performed by the team or group preparing the plan, not by outside critics. The methods are explicit and reproducible by others, not idiosyncratic. Analytic devices such as cost-benefit analysis or goals achievement analysis (Hill 1968) are used, not implicit criteria dependent upon the critic's individual "virtualities." And the evaluation process has several stages (e.g., phases 7, 8, and 11 in Lichfield, Kettle and Whitbread's 11-stage schema) where these analyses can be used. In all instances, it is the substance of the alternatives, only, that is at issue--not the overall plan.

Kaiser, Godschalk, and Chapin (1995) also stress evaluation by the plan's preparers during its formulation (what they call "pre-adoption evaluation") so as to understand the potential consequences of alternative proposed plans. The effects include both total effects and the more disaggregated distributional effects. Kaiser, Godshalk, and Chapin also speak of plan monitoring, a continuing assessment of the consequences stemming from the plan (not shown in figure 1, for simplicity).

Comparative Plans Research and Professional Evaluation.Another type of evaluation occurs after plan adoption (either before outcomes can be

evaluated, or with outcomes not intended to be part of theevaluation). Although in spirit it resembles the critic's effort, the methodology is quite different. Here the evaluator is a researcher, either one inside the organization that prepares the plan, and who is trained in how to improve subsequent plan effort, or an outside "pure" researcher, perhaps trained in some discipline other than planning. Usually several plans are compared systematically. One of the early research comparisons was by Gruft and Gutstein (1972). To establish their criteria, they assumed that: (1) planning should be a rational process; (2) it should be a democratic decision-making process; and (3) plans(and planning reports) should add to the body of knowledge about urban design. Under these three headings, sixteen more specific criteria were included. Applying their criteria to eighteen "representative" reports, the authors found that most of them were filled with circular reasoning, preconceived solutions and self-fulfilling prophesy, in which it was apparent that problems and objectives had been determined and data selected after the solution was designed ... [and] ... many decisions were based on popularizations of social science theory, superficial analogy and doubtful rule of thumb.... [T]he necessity for the participation of citizens in decisions which would affect their lives was almost ignored. (Gruft and Gutstein 1972, 26-2-3)

The Gruft and Gutstein (1972) pioneer evaluation was comparative, but looked only at the plan document, not at plan outcomes. Recently there have been a number of comparative evaluations. Connerly (1990) for instance, examined the quality of housing elements in plans; Stiftel and Boswell (1994) compared the treatment of uncertainty and the time horizons, public involvement, and intergenerational equity in eight general plans in Florida. In the most thorough and methodologically self-conscious research evaluation yet, Berke and French (1994) and Dalton and Burby (1994) (see also Burby and Dalton 1994) examined natural hazard mitigation in 139 local comprehensive plans, in five states. Although using a limited number of criteria for plan quality, their evaluations were substantially more systematic than earlier efforts, and their methodology serves as an exemplar of research and professional evaluation. All the plans were "double coded" (two different people coding the same plan) and on an ordinal scale (whereas most criteria use only a nominal scale), so that statistical tests could be performed in connection with the plans' quality as expressed in plan documents.

Evaluating Post Hoc Plan Outcomes. After plans are adopted and implemented, they can be further evaluated, empirically, in terms of their outcomes. The purpose is usually to discover if the plan was implemented, and if so, how it performed (Alterman and Hill 1978) or what its effectiveness was (Faludi 1987, 127). (A related history and literature is found in the field of policy implementation [Alexander and Faludi 1989], which compares policy and program intention with actual result or outcome). Both literatures gloss over two fundamental criteria: when should the outcome be determined? and what terms should its performance or effectiveness be cast in, that is, what should the actual outcome be compared to?

Plans usually show the time needed for the plan to be fully implemented (e.g., 10, 15, or 20 years). But they never show how much time should elapse before the full effects of the plan should be evaluated. Surely a 20-year plan should not have its outcomes fully evaluated after, say, only five years. Krohe Jr. (1993), for instance, provides a nice sketch of the various effects from Burnham's plan for Chicago during the last 84 years.

Even if we can decide when, after implementation, outcome evaluation becomes appropriate, what alternatives should we compare, and what does any difference revealed actually mean? Typically, planners and policy analysts have assumed a "blueprint" mindset, comparing the plan's intended outcome against what happened. There are two objections to this. First, if the profession no longer views plan-making as the planner's sole function, but sees planning process as more important than any plan, then a question arises as to what it means to evaluate plans post hoc on the premise that any difference between reality and the plan reflects poorly on the plan alone. If process is the profession's overriding concern, then plans should be viewed as simply an important and necessary device for working out the future, but not a sufficient device. Faludi (1987) makes this point most forcefully: "Of course, plans are departed from! So what? Let's get rid of the odour that hangs over departures." Yet invoking process to cover contingency and uncertainty makes it extraordinarily difficult to anchor any basis on which to judge plan quality, which is afloat in a sea of relativity (Alexander and Faludi 1989). We should be mindful of and calculate departures from the plan, but we should not despair at the existence of departures.

There is a second issue: should the post hoc measure be the difference between plan and reality, or the difference between what would have occurred in the absence of any plan and what happened with a planin place? Discouragement due to the plan not being fully implemented might be called the half-empty-glass syndrome; elation that reality turned out to be more like the plan than it would have been with no planmight be the half-full-glass syndrome.

The post hoc evaluator must be clear about the purpose of the investigation and the criteria for outcome evaluation, that is, what was expected versus what happened. But with that in mind, we can consider several variants in post hoc evaluation, which depend upon the underlying conception of the plan. These and the points discussed above are illustrated in figure 2, a continuation of figure 1.

The various permutations of post hoc evaluation are described below.* Reality (the actual outcomes) can be compared to the expected outcomes if there had been no plan (the null case).* Reality (the actual outcomes) can be compared against the intended outcomes from the plan. This approach assumes that the plan is a blueprint and any departure reveals its weaknesses. Moreover, it assumes that the plan is definite and precise. As Alterman and Hill (1978) point out, any flexibility built into the plan makes a determination of the "true" plan intent or outcome more problematical.* Reality can be compared against the plan, but rather than focusing on the difference between the two, the focus can be Litchfieldian, on what difference any unanticipated consequences from the plan actually made.* In a Faludian conception of the plan, reality can be presumed to be only loosely linked to it, since there are so many possible reasons besides the plan to account for an outcome.(FN2)* In a postmodern conception of the plan: where does one look for an outcome if the plan itself is conceived as symbolic or expressive or merely a way station to the larger goal of achieving dialogue about community? For instance, is it the physical outcome that is examined, or the change in the sense of the community's alienation? More fundamentally, is the notion of "substantive outcome" misplaced? Is the appropriate outcome to be measured actually the change in the community agenda after plan dialogue and discussion, as the appropriate focus of post hoc evaluation?(FN3)

A REVIEW OF PLAN CONCEPTS AND THEIR ASSOCIATED CRITERIAThe appropriate criteria to evaluate a plan are implicit in the concept that the plan embodies.

Moreover, in reciprocal fashion, the plan's concept is clarified only by considering the criteria to judge it. For instance, a vision plan implies different criteria than for a blueprint plan, and a symbolic or expressive plan, different criteria than for an instrumental one. Each time a plan is prepared, the preparers should specify the criteria by which they expect it to be evaluated; the effort inevitably sharpens their understanding of what they are about.

The following review of plan types emphasizes the appropriate criteria in light of the particular concept of plan. The purpose is to provide background for judging the evaluation criteria at the "plan assessment" stage presented later on, in figure 3.

The Plan as Vision. The plan as vision has once again garnered popularity in the profession (Klein et al. 1993). Simply put, the vision plan is an elegant attempt to publicly propose a "What if?"—to stimulate thought and elicit comment. Appropriate criteria pertain to communicating the vision in an empathetic, visceral and stimulating way that touches not just the mind, but the soul. The reader should be attracted into the exercise, lifted to the prospect envisioned, convinced as to its possibility (or that of one like it), and provided just enough "realism" to convince the natural skeptic in us all to at least momentarily suspend disbelief.

The Plan as Blueprint. There was a tendency in the early days of planning to believe that a vision plan could readily be converted to a blueprint plan. The early term "Master Plan" nicely captures the underlying blueprint sentiment. The physical orientation of a Master Plan as blueprint was reflected in Bassett's (1938) early call to limit the scope of the plan to what can be shown on a municipal map: elements like streets, sidewalks, street trees, parks, lakes, public building sites, and the general districts for which private land should be zoned.

If plans were to be like blueprints, then criteria had to expect that each and every aspect of the plan as well as their interconnections would be described or mapped or diagrammed in considerable detail, to demonstrate mastery of all the plan's aspects and how they joined. To ensure sedulous implementation (for that is the point of the blueprint), criteria are required for each control point in the system for implementation à la Ashby's Law of Requisite Variety (Ashby 1958).

Unfortunately, this approach produced few plan successes in the "real world." Often it was the zoning ordinance that was really the blueprint, while the general plan, if there was one, was strictly advisory. Many in the profession who had hoped for more concrete results turned to alternative routes for plan success, for example, urban renewal plans; others eschewed the need for a plan altogether, preferring to emphasize planning processes.

The Plan as Land Use Guide. The general plan as land use guide appeared in the 1950s. Although there were variants, it took a middle of the road approach to what a plan might be; thus its criteria were as follows:

The plan was a vision of the future, but not a blueprint; a policy statement, but not a program of action; a formulation of goals, but not schedules, priorities, or cost estimates. It was to be uninhibited by short-term practical considerations. (Kaiser, Godschalk, and Chapin 1995, 369)

Kaiser, Godschalk, and Chapin (1995) go on to distinguish four modern branches stemming from this earlier general plan. Borrowing from concepts in other plans of the 1960s and 1970s, as discussed below, all four types stress more action and implementation, as well as more citizen participation, than those earlier models did. There are land use design plans (detailed mapping of future land uses with some action strategies); land classification plans (non-parcel-specific, and more frequently found at the county or metropolitan level); verbal policy plans (verbal policy and action with no maps); and development management plans (maps and very specific implementation tools and programs for action, including location, type, and pace of growth).

The Plan as Remedy. The alternative to the plan as vision is the plan as remedy: the cure for an existing problem. Urban slums and the promise of federal moneys for their renewal in the 1950s provided suchan alternate route to plan reconceptualization. Redevelopment plans were one result. Unlike general plans, redevelopment plans were short-range and specific, aimed at remedying the problems of slum housing, not at building new housing in the suburbs. The appropriate criteria here were largely federally determined, and were in the form of hurdles to surmount, or tests to pass, in order to receive a grant of funds: Local governments had to demonstrate their need for renewal by producing a comprehensive description of slum problems, and demonstrate their administrative and

financial capacity to respond effectively by citing a Workable Program that called for a general plan, building codes, financial capacity--and, for the first time--citizen participation in governance.

The need to sell vacant redevelopment land introduced planners to the criteria of property valuation, economic and financial feasibility, and market analysis. Short-run market and political objectives also introduced planners to the criteria of steep discount rates, by which project-delimited urban renewal plans or even rezoning requests outranked comprehensive long-term planning goals (Fischman 1978, 372-3).

The scope of remedial planning encompassed ever larger perspectives with the subsequent General Neighborhood Renewal Plan (GNRP), which focused on a sequence of renewal projects in a larger neighborhood, and still later the Community Renewal Plans (CRP), which scheduled revitalization for the entire city. Several new kinds of criteria were thus implied. These might be called programming criteria, because they emphasized implementation timelines, and stressed coordination between urban renewal (existing, scheduled, and proposed) and other local public expenditures within the city.

The Plan as Administrative Requirement for Federal Funds. Other kinds of federally supported plans eschewed urban renewal's geographic focus for a substantive or functional one. The prospect of federal funds for what had traditionally been nonplanning areas quickly led in the 1960s to health facility plans, transportation plans, employment plans, housing plans, educational plans, juvenile delinquency plans and community action plans. The Model Cities' social/administrative "plans" were the culmination of this relatively nonphysical, federally funded comprehensiveness. The "systems approach" to planning and itscriteria for monitoring and controlling complex interaction were sometimes emphasized here (Robinson 1972, 423-518), as was the administrative capacity to carry out such complex plans. HUD used suchcriteria as "city commitment," "administrative capability," "understanding of problems," "clarity and reasonableness of goals and program approaches," "degree of innovation," and "capacity building" to evaluatethe plans submitted for funding (Frieden and Kaplan 1975, 132-4).

The types of criteria added to planning in this era were many. The wide-ranging concerns led to a greater appreciation of, and a longer list of criteria for what being comprehensive really meant. Each substantive or functional area plan embodied its own criteria based on social science findings and paradigms. Other plan criteria were grounded in legal rules for intergovernmental loans and grants, requirements of the bond market, and political and administrative theories of intergovernmental relations.

The federal government encouraged regional plans, as well. The Office of Management and Budget (OMB) issued its Circular A-95, areawide clearing-house procedures for Councils of Governments (COGs). These councils were to coordinate on the regional level the various federally-assisted projects planned at the local level (Code of Federal Regulations). Projects were to be designed and located "in accordance" with the regional plan.

Process, not Plans. Beginning in the 1960s, planning as an ongoing process, rather than merely making particular plans, was widely embraced as the profession's true purpose. As Altshuler (1965, 98, 130) noted, a common planning principle held that "planning is more important than any plan." Moreover, the technical and factual underpinnings of physical plans were deemed insufficient. The city was no longer thought of as simply an artifact to be shaped by criteria for physical design; it was viewed as a mechanism of interrelated parts to be selectively transformed by applying criteria derived from systems theory. The social sciences were introduced to equip planners with more understanding of urban cause-effect relationships. Modeling urban processes, not drawing up a blueprint, was the key component shaping thecriteria for this system (McLoughlin 1969).

Two other facets of a process orientation were citizen participation and planner advocacy for the poor. These concerns, too, introduced their own criteria. And in broadening the planner's perspective, social science also cast doubt on the validity of traditional planning criteria. There was no unitary "public interest," the social sciences reported, nor any possibility of rationalism and comprehensiveness. 'Satisficing', and 'disjointed incrementalism' were put forward as more realistic views (Simon 1957; Braybrooke and Lindblom 1963), although it was not proposed that they be criteria as well. Planning's legitimacy could be bestowed only by a participatory or advocacy process. Perhaps the only real use of a plan in this view was as a community-based counter plan to one proposed by city hall.

With this emphasis on the process of plan making, many professionals came to view plans as a side-show, the main event being the larger processes that encompassed plans along with other planning activities. Hence, criteria for plans were deemed unimportant. Catanese (1974), however, questioned the wisdom of emphasizing process over plans. Process, he said, might be the theoretician's chief concern, but it was mistrusted by many practitioners and by much of the public. What criteria can be invoked for the planning process? he asked. How are planners to be held accountable for their recommendations, and how are politicians to be held accountable for their decisions, if all planning is continual process?

Plans as Pragmatic Action. Discussion about plans during the 1970s tended toward the pragmatic. The American Law Institute's (1976) extensive Model Land Development Code, directed toward revising the enabling laws for planning, provided a streamlined framework for subdivision, zoning, and redevelopment. Its implicit criteria were procedural, focusing on how to mesh these assorted devices, but paying less heed to concern for the plan as overall direction-setter for development (Hagman and Juergensmeyer 1986).

Other planners also thought state plan-making legislation should be revised. Some argued that states should mandate local plans, because the criteria for fair land use decisions demanded it (Mandelker 1978); others argued that states should not mandate local plans, because any implied criteria for consistency are unfeasible, their costs unfundable, and their local support negligible (Susskind 1978). California legislation embodied yet other practical criteria: the number of substantive elements to be either required or allowed in the general plan as governing its content (Governor's Office of Planning and Research 1990a).(FN4)

Policy plans were introduced in the 1970s, to meet the social and economic concerns that had been omitted in the so-called comprehensive plans for physical development of the 1950s. These plans, however, often deliberately omitted the land use element and the accompanying criteria for maps and diagrams, preferring a verbal set of policies organized according to functional systems (Macris 1994). Kaiser, Godschalk, and Chapin (1995) noted that the lack of maps in policy plans embodies one of their prime criteria--geographic flexibility, and that they also reflect criteria of appropriate scale that apply, especially at the state level, where land use maps may be inappropriate. Cleveland's Policy Plan, on a different tack, emphasized economic redistribution; it became an exemplar of a radical plan designed within, yet clearly stretching, the criteria imposed by a conventional institutional context (Krumholz, Cogger, and Linner 1975).

Plans as Responses to State and Federal Planning Mandates. During the 1970s, state-wide plans came into being, often triggered by environmental concerns (Wilson, Tabas, and Henneman 1979; DeGrove 1993). The California Coastal Plan, drawing heavily on geology, biology, and other natural sciences, was an early example that revealed the potential authority of the natural sciences in a plan and demonstrated that environmental effects were powerful evaluation criteria. Most recently, in this vein, habitat conservation plans (HCPs) have introduced substantive criteria based on biodiversity and ecology (Beatley 1994).

Oregon produced the first comprehensive state plan, with explicit goals to direct state agencies and local governments in their land use decisions. In general, the state mandate process was top-down and hierarchical, and invoked criteria relying on command and control (Buschbaum and Smith 1993). Each city's independent preparation of its plan, as assumed by the early advocates of the general plan, gave way to criteria of intergovernmental coordination. To the traditional criteria of internal consistency were added state requirements for vertical consistency between plans of these state agencies and those of local government (Oregon, New Jersey, and Florida); for horizontal consistency, or agreements between communities (called "cross-acceptance" in New Jersey); and for temporal consistency (called "concurrency" in Florida), ensuring that infrastructure development not lag behind new private development.

California enacted so many requirements for ancillary plans to fit within the general plan that it implicitly created the general plan as switchboard. Criteria now involved interplan compatibility, so that the more narrowly designed technical plans could "plug into" the General Plan, and so connect to each other as well (Governor's Office of Planning and Research (1990a).(FN5) Compatibility between different state-mandated technical plans suggests a new major criterion for evaluation, that of transaction costs (Alexander 1992).

At the federal level, other administrative mandates were introduced that reshaped the plan. Brooks (1979) described the result as the "administrative regulatory model," reflecting federal requirements to planfor clean air and water (State Implementation Plans [SIPs]). Key criteria for these plans were that they show how the plan would be implemented and who was responsible for implementation.

CRITERIA FOR EVALUATING THE PLAN AS EMBODIED BY THE DOCUMENTPLAN ASSESSMENT WHILE PREPARING THE PLAN

The discussion in the previous section was--except for plan testing—about evaluations after a plan has been formulated. Another kind of "evaluation" to be examined here is carried on while the plan is being formulated. To distinguish it from testing and evaluating the merits of substantive plan alternatives as discussed by Litchfield, Kettle, and Whitebread (1975) or by Kaiser, Godschalk, and Chapin (1995), we can call this step plan assessment, an assay of the plan in process. Plan assessment evaluates professional expertise: the plan as a document communicating methodology, reasoning, and content; it operates at the moment of bringing a plan into being. (The stage for plan assessment is shown in figure 1.)

Criteria are needed for what the plan as a document should include, and how its professional competence should be judged. At minimum, these criteria should guard against serious errors of omissions, in a fail-safe approach.(FN6) A minimalist way to do that is to set down the criteria for a bad plan. Hansen (1968) argued that one can have little respect for a plan characterized by: unavailability, pedestrian style, obsolescence, incomprehensibility, unreality, or neglect. But a profession needs criteria beyond mere admonition. Positive criteria should be available that specify what plans should have to meet professionally approved standards of practice analogous to the accounting profession's "generally accepted accounting standards." One example is California's PLATO (Governor's Office of Planning and Research, 1990b), a set of plan criteria, and the related 346-page guidelines for general plans (Governor's Office of Planning and Research, 1990a), from which still more criteria can be derived.

Criteria for the plan as a document also would respond to the changed legal expectations about planners' competence and skills. The United States Supreme Court and many state courts are signaling planners that there must be better documentation of what they propose and why (Nolan v. California Coastal Commission 483 US 825; Dolan v. City of Tigard 129 L. Ed 2d 304 at 320 and 323 [1994]). Increasingly, traceable "footprints" are required to show how planners got from their description of the current situation to what they recommend for the future (Governor's Office of Planning and Research 1989). Mere assertion that the plan is in the public interest must be replaced with evidence that it is.

PLAN ASSESSMENT OVER TIMEAnother approach to plan assessment is, like post hoc evaluations, comparative, but

comparative over time, asking: Has the state of the art of plan-making improved?(FN7) It would seem so. The four editions of the classic planning text, written by Chapin in 1957 and revised by him in 1965, with Kaiser in 1979, and with Kaiser and Godschalk in 1995, demonstrate how plan-making has changed, for the text has always been at the forefront of planning practice, nicely capturing a sophisticated view of the respective eras. Kaiser, Godschalk, and Chapin's 1995 edition reflects a planning practice substantially bettergrounded than the practice viewed in the original 1957 edition. Presumably the profession's plans would fare better under Gruft and Gutstein's criteria today, some 20 years after their formulation. On the other hand, today we might expect the criteria to have improved as well. Stiftel and Boswell's (1994) first three criteria: (1) time horizon (e.g., discount rate); (2) risk analysis (e.g., types of uncertainty considered); and (3) intergenerational equity would not have been posed in earlier days.

PLAN CRITERIA--A FIRST CUTIt is time to present some specific criteria. A list of plan criteria is boring in the same way a

list of household tasks is. Criteria are most useful and interesting when one is actually preparing or evaluating a plan. Therefore, instead of presenting a series of suggested lists, I have gathered lists of

partial criteria from the literature and formed them into a composite list (figure 3) of about 60 items, grouped into eight basic classifications.

The criteria are not definitive. They are advisory and suggestive. They are intended both as a check list of possible considerations during plan preparation, and to stimulate plan authors to devise variations and additions they find more pertinent to their particular plan. The aim is to generate formal, self-conscious reflection on what is being done in plan-making, while it is being done.

POSSIBLE OBJECTIONS TO USING CRITERIAMODERNIST OBJECTIONS

There are several possible objections to using criteria, and/or these in particular. First, it might be argued that the criteria listed in figure 3 are for the most part restricted to internal professional reviews: either as self-testing criteria, or as criteria that one professional would use to judge the work of another.(FN8) Indeed, other types of criteria are possible, based on, say, political accountability or the self-interest of different groups; these criteria, however, should be (and will be) developedby nonplanners.

A second objection: clarity, explicitness, and a generally well articulated and documented plan under the criteria found in figure 3 may nevertheless have little effect in the real world (Dror 1968). This critique reflects the view that "things are not as they seem," that says "What is really happening is ...," and that goes on to invoke a conspiracy or cabal, or to deplore the obvious mystification employed by the plan todisguise what really transpired. Plans as conspiracies are hard to disprove. Every subsequent event is said to reinforce the charge. Even those bits of evidence that would seem to refute the charge of a conspiracy are used to demonstrate how insidious and subtle the plot really is.

Third: technical quality, the characteristic emphasized here, is merely one approach to doing better. Other reckonings may find criteria and principles of technique to be the planning problem, rather than the planning solution. There are two reasons.

1) Formal, legal criteria can become a litigious nightmare. Criteria become hooks upon which to drape all kinds of cloaks for all kinds of reasons, many of them having nothing to do with "good" plans, but rather with narrow goals of interest groups. The Environmental Impact Review process in California is a good example of checklists and criteria being subverted for untoward ends.

2) Technical failings such as vagueness, ambiguity, or even inconsistency and generality may foster a plan's success. Frequently these very qualities meet the political necessities for a plan's survival (Altshuler 1965, 132-3).

Unfortunately, an outsider evaluating a plan cannot discern whether its technical inadequacies were deliberate, to satisfy political demands, or were the result of professional incompetence. Worse, again for political reasons, the plan cannot even refer to the political criteria it follows. To politicians, "dumb" planners are often a blessing; indeed, what the plan does not say may be more important than its explicitcriteria (Milroy 1991). These kinds of concerns lie beyond professional criteria, responding to meta-criteria of professional norms, values, and ethics (Fischer and Forester 1987).

POSTMODERNIST OBJECTIONSThe postmodernist critique of plans seems to oppose criteria like those outlined here, and is

probably even more opposed to those imposed by state mandates. Both types assume the conventional expectations of all professions: a mastery of a technical body of knowledge and praxis to be used in furthering the public interest. Krieger's (1981) critique, for instance, attempts to peel away the scientistic veneer of plans by reminding us that plans are, after all, human creations; for all their objective and putatively scientific trappings, they are really only "stories." Some postmodernists would go further, questioning the very presumption of technocratic expertise in figure 3, and claiming that such expertise should be demystified; that plans should be viewed only as parts of more basic discourses and arguments; that an orientation to rhetorical skills is at least as important as technique; that plans are a means of expressing values and belief systems rather than technical pronouncements of cause and effect (e.g., Fischer and Forester 1987, 1993).

In this light, plans increasingly came to be viewed as vehicles for reasoned argument rather than as depositories of scientific and rational findings (Fischer and Forrester 1993). Mandelbaum

(1990) suggested ways of considering a plan: as a policy claim (a system for command and control with inputs and outputs); as a design opportunity (a response to a problem or crisis); or as a story (a narrative that imposes structure, with beginnings and endings, upon what is really an endless flow of events). Throgmorton (1993) emphasized the plan's inevitable rhetorical aspects and activities, constituting attention to its audience (and in the process of writing for one audience, perhaps creating another). He also noted the plan's unpredictable play of meaning despite the expert's stance of solitary objectivity, and emphasized that this aspect, however discomfiting to some planners, is inevitable in all plans. Others have echoed the notion of plans as "texts to be read":

Are they scientifical [sic] reports, i.e. discussions of some related empirical evidence? Or codes, i.e. systems of norms and commands? or political perorations, i.e. rhetorical praises of some definition of the public good? Or dramas, i.e. sequences of accidents and solutions, each having its turn? (Ferraro 1994, 210; emphasis added)

Specific criteria for plans from the postmodern perspective are only beginning to emerge, for example, Healy's (1993) criteria of discourse and discourse communities, communicative work, and power relations. Such postmodernists clearly invoke different dimensions of criteria than the conventional ones discussed above; indeed, at times they seem to deny the feasibility and legitimacy of the conventional criteria. Consider the following postmodernist criteria:

The texts of the postmodernist planner, in fact, should be consciously fragmented and contingent, nonlinear, without aspiration to comprehensiveness, singularity, or even compelling authority. This clearly contrasts with the master plan of modernist planning; it is meant to be cohesive and visionary. (Beauregard 1991,192)

Plans, in the postmodern view, can't be comprehensive and internally consistent, given the myriad voices in the community. Plans cannot always be expected to "do things"; their function often turns out rather to be expressing and legitimating beliefs to make those in control feel meritorious. Plans should facilitate doing the right thing, but the rightness, if there is one, may emerge only in the course of the "argument" of which the plan is but a part.

Surprisingly, Jack Kent (1964), an early general plan theorist ignored for thirty years by other planning theorists, had in the 1960s anticipated many of these salient issues of the 1990s. For instance, he emphasized such criteria as the need to decide who the client of the plan is (the issue of multiple voices). For Kent, the client was the City Council rather than the Planning Commission or the City Manager. He then discussed how the city council should use the plan (the "argument" issue), claiming it should be for legislative rather than advisory or administrative use. He said that the five legislative uses of the plan (meta-criteria, perhaps?) were for (1) policy determination; (2) policy effectuation; (3) communication of these policies to the community; (4) exchange of information pertinent to planning concerns (conveyance of advice); (5) increasing the community's knowledge about planning problems and prospects (education).

This orientation led him to recommend (in anticipation of the "rhetoric" norm) that the plan should be in a form suitable for debate. Indeed, he argued that the alternatives considered and rejected in the plantesting phase should nonetheless be kept in some fashion, so that later citizens could learn and consider what the choices had been (Kent 1964, 89). Kent's modernist answers to his postmodern question may not satisfy postmodernists, but the issues he raised suggest that even modernist planners, as well as crafters of state planning mandates, cannot escape postmodern questions. If these questions cannot be escaped, do postmodern criteria belong in state mandates? The issue will be discussed in the article's conclusion.

CONCLUSIONThe emphasis here has been on the plan as a product, not the plan as an incidental output of

the planning process. Evaluation criteria for plans are important in both the several aspects of plan-making and in post hoc evaluation. I have paid special attention to plan assessment, and the criteria to be used in the course of making and assessing the plan. Criteria should not be viewed merely as a checklist, nor received unquestioningly from on high, whether as state mandates or some currently fashionable theory. Instead, planners should devise their own criteria in the course of making the plan. A planner's skill should include the ability to formulate criteria as well as to prepare goals and objectives for a plan.

Yet what kind of criteria should be used? The series of modernist objections discussed in the previous section ineluctably lead to the type of arguments raised by the postmodernists. If postmodern views are true even in part--and it would be hard to wholly deny them--they raise troubling issues for state mandates. Are these state mandates beside the point? Should postmodern criteria be urged as substitutes or as complements? Can postmodern criteria be operationalized? What would be the effect, for instance, if state mandates required something operationally postmodern like the following?

1. The plan shall be both instrumental and expressive. In the event of conflict the latter is paramount. Plan policies should be sufficiently ambiguous to obfuscate the inherent contradictions in society.

2. The plan may be fragmented and nonlinear, constituting one of many possible stories rather than a definite scientifically-based finding or an account of cause and effect. The plan in its particulars should hold up a mirror to the disjointed nature of society.

The examples are no doubt strained, but they point up a weakness when postmodernism moves from critique to recommendation. Postmodern values are easily abused in the name of discourse. Planners, bewildered by postmodern "weightlessness," could readily develop criteria that are Machiavellian applications, similar to the modern techniques of "spin doctors." Planners who become skilled in rhetoric could easily become like the rhetoricians of ancient history, sophists constructing political perorations--or, worse, political pirouettes. And what can citizens expect from a postmodern mandate after politicians learnto "interpret" the mandate in postmodern ways, indeed learn to "deconstruct" their local plans? Postmodernism's "multiple vocalities" can become a rationalization for a wide variety of political postures and duplicitous actions. Such cynicism is not what postmodern values intend, but these consequences are likely even if unintended.

The criteria presented in figure 3 are one answer to what is still a modernist world, if hemmed in by some postmodern cautions. The critera are explicit; the results of applying them will be relatively self-evident; the necessary corrections to a plan to meet the criteria are relatively easy to make. And the results are understandable to politician and citizen alike. In these respects the modernist criteria reflect all 20th-century professions' belief that professional criteria can be systematized, codified, professionally approved (like accounting standards) and helpful to the public.

Moreover, present-day legislative mandates pose a special dilemma that these criteria can help to resolve. At the moment, state planning mandates are frequently a curious amalgam of incomplete views of theplan, joined with assorted criteria engendered by interest groups uninformed about the relationship between plan concept and criteria. Planners attempting to meet state mandates find themselves both devising the plan concepts that the state-mandated criteria suggest but could not articulate, their authors not being experienced in planning theory; and also inventing the plan evaluation criteria that the incomplete state-mandated plan concept implies. Focusing on plan evaluation criteria and their relationship to the various stages in making plans, as suggested here, should allow planners to deal better with the naive enthusiasm that frequently gets incorporated into state mandates. These evaluation criteria will remain helpful to planners so long as they do not become totally enamored with the technique and expertise that the criteriaimply.

Added materialBaer is a professor of urban planning and development at the School of Urban Planning and

Development, University of Southern California. He is currently engaged in research on the continuum that ranges from constitutions, to plans, to regulations, to standards.

The planning profession has developed relatively few criteria for evaluating the quality of general plans. Evaluation criteria have become more important with the increasing number of states that mandate general plans. Several kinds of plan evaluation exist, and these are distinguished and described before a review of different concepts of plans as a source of the appropriate criteria to evaluate them. A list of suggested criteria for plan evaluation during plan preparation is then presented, to be used to make the plan better. Appropriate criteria for a plan are not easy to devise, and the postmodern critique of planning makes this task more difficult still. These issues are explored as well.

AUTHOR'S NOTEI am grateful for the indulgence of three anonymous reviewers who put up with considerable imperfections in an earlier draft while offering enormously helpful comments. Niraj Verma and Chris Williamson also made helpful comments on an earlier draft. This article is a substantial revision of an uncompleted draft, "Plan Appraisal: An Approach to Planning Practice and Education," written 20 years ago with my colleague, Ira Robinson. Tridib Banerjee made useful comments on this earlier effort as well. I am, of course, responsible for any errors.

FIGURE 1. Various stages for evaluation in the planning process(Planning process model adapted from Alexander 1992)FIGURE 2. Post hoc plan evaluation(Planning process model adapted from Alexander [1992])

FOOTNOTES

1. See, for instance: Berger (1967); Hill, 1967; Villamil (1967); Review Forum (1970); Schneider (1972).

2. Alexander and Faludi (1989) provide a richer discussion of the situation in the 2-4 instances.3. Tridib Banerjee made a number of perceptive observations about the plan in a review

of an earlier draft of this paper, when he noted that when a plan--especially a general plan--is prepared, it sometimes serves as a source book for the community, a kind of "yellow pages," or community manual of social information. Accordingly, the plan in historical retrospect can serve as the "minutes" of an era, recording both objective aspects of a community and its normative concerns as well.

4. Examples of these elements include: circulation, open space, housing, seismic safety, and noise.5. California has requirements for: a coastal zone plan, specific plans, congestion

management plans, noise management plans and airport land use plans (all mandated), among others.

6. Criteria provide some administrative benefits as well. They help ensure that taxpayers' money is well spent for planning, that local elected officials can assess the relevancy of proposed plans and their expected payoffs, and that citizens and community leaders can also accurately evaluate the entire work program and individual work elements (Thomas 1971).

7. Keene and Strong (1970) for instance, in describing the making of the Brandywine Plan, presented a technical elegance matched by few others of that era. However, they also said that the plan "with its heavy baggage of legal, hydrological, financial and research goals was probably overloaded," and add that these technical attributes may have contributed to its defeat. Today, the same baggage might not seem so heavy. City councils and the public are now much more technically sophisticated, willing to follow planners and other experts in their technical labyrinths.

8. I am grateful to one of the reviewers for pointing this out to me.

FIGURE 3. Some suggested general criteria for plan assessmentAdequacy of Context. (Explain the context and setting: the what and the why of the document. They are not self-evident to the public.)

1. Is the political/legal context of the plan explained (e.g., meeting state mandates, public discussion and consideration, top priority issues)?

2. Is the administrative authority for preparation indicated (Council or Planning Commission resolution, state law, federal requirement, etc.)?

3. Is the role of the preparing agency or firm adequately explained (e.g., a letter of transmittal)?4. Is background information presented (e.g., reasons for plan's presentation)?5. It is clear who the plan is for (e.g., citizens, agency head, city council, board)?6. Is the purpose of the plan explained (e.g., study, information, decision, action, conveyance of

advice)?

7. Is the type of plan and its scope reported early on, to alert the reader about what to expect? (E.g., the reader is alerted that this plan is highly quantitative and analytic; far ranging or narrow; specific, and technical.)

8. Is an overview/summary provided (e.g., an "Executive Summary")?9. Is the source of funding for the plan shown (e.g., federal, state, local, private donor, agency)?10. Is the amount of time in preparation shown (total person/hrs., weeks, etc.)?

"Rational Model" Considerations. (Show basic planning considerations based on underlying theory and its criteria. Even beyond the list here, there are many theories and types of plans. The plan authors must be clear themselves about what they are doing, to transmit clarity to the reader).1. Given the type of plan to be prepared, are the plan formulators clear about the criteria they

will use to assess its progress while being formulated?2. Have these criteria been made explicit in the plan?3. Are problems specifically identified (or only implied)?4. Are goals and objectives explicitly identified?5. Is the tone of the plan commensurate with the planning approach recommended (e.g.,

comprehensive, incremental, advocacy, etc.)?a) If the plan is intended to be comprehensive, does it relate substantively to a larger

whole (e.g., horizontal relation to other agencies and adjacent governing bodies)?b) Does the plan consider the regional or next higher level of government or context

(e.g., vertical relation)?c) Is there planning for procedural coordination with other plans and agencies?

6. Is the capacity or adequacy of existing infrastructure and organizational systems described?7. Are alternatives listed, or at least considered?8. Are the alternatives identified as "variations on a theme," or as radically different?9. Are tradeoffs permitted?

Procedural Validity. (Explain the who and the how of the plan-making; inform the reader about what went on in making the plan and what is going on by publishing it.)1. Who was involved in the plan formulation (e.g., staff from different agencies or departments,

citizen groups, politicians)?2. How were they chosen (e.g., on the basis of expertise, interest, volunteering, or other self-

selection)?3. How were they involved (e.g., discussion groups, internal staff memos or papers, public

meetings)?4. How were data, models, goals, and other pertinent information used in recommending policy

or action?5. How were technical matters transformed into recommended policy (e.g., through "ordinary

knowledge," experience, "scientific" training, design training)?6. Was an advisory group used?7. Were preliminary drafts circulated for public comment?

Adequacy of Scope. (Show how the plan is connected to the larger world.)1. Have all possible or pertinent issues been considered (e.g., physical, social, economic, political,

psychological, cultural, or design)?2. Have issues of efficiency and equity and predictability been considered?3. Has the distribution of costs and benefits among different groups and interests been considered?4. Have relocation/displacement implications been considered?5. Have financial/fiscal implications been considered?6. Have the legal implications been considered?7. Has feasibility in the larger political context been considered?

Guidance for Implementation. (Most plans are intended to do something. Consider the instruments [ordinances, regulations, budgets, schedules, etc.] and the agencies and persons responsible for making theplan work. Should they be included? [A vision plan would not have an implementation aspect; rather, it would have a section dealing with "the next steps."])

1. Are implementation provisions appropriate in the plan?2. Are there priorities for implementation?3. Is cost of implementation vs. nonimplementation considered?4. Is there a time span for plan implementation?5. Is there povision for scheduling and coordinating of implementation proposals?6. Can proposals accomplish their intended purpose if implemented?7. Is there a program or proposal for an impact analysis?8. Is the agency or person responsible for implementation identified?9. Can the responsible agency realistically be expected to implement the plan?

Approach, Data, and Methodology. (Make clear the technical bases, if any, of the plan; where the data come from and how they are used, so that others may check the plan's thinking by use of the same sources.)1. Is the plan based on a wide spectrum of data where feasible?2. Is the plan sufficiently flexible to permit new data and findings to be fed in?3. Are the data sources cited?4. Are the methodology sources cited?5. Are the levels of data aggregation relevant or meaningful to the study?

Quality of Communication. (Clear communication above all else is necessary for a fair hearing from others.)1. Is the client or reading public identified (e.g., public at large, other professionals)?2. Are the ideas convincingly presented, given the nature of the audience?3. Are the rationales behind the decisions effectively presented?4. Are the proposals/recommendations/conclusions consistent with objectives?5. Is the tone of the document consistent with the message conveyed (e.g., not presented in the

past tense as an accomplished fact when the plan is for study and review)?6. Are the criteria indicated by which the plan is intended to be judged?

Plan Format. (Other forms of communication are found in the plan format itself, as well as evidence on who takes professional responsibility for the plan's formulation, when it was adopted, and other seemingly incidental concerns that nevertheless communicate professional competence.)1. Are the size and format conducive to the use intended? (For example, an oversize plan is

hard to file and copy, hence does not lend itself to constant reference and day-to-day use.)2. Is the date of publication shown?3. Are the authors shown, to indicate professional responsibility (names of personnel who

worked on the plan, as well as agency or firm names)?4. Is there a table of contents?5. Are pages numbered?6. Are graphics used to best advantage?7. Is the plan attractively laid out?