IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. SMC-14, NO.1, JANUARY/FEBRUARY 1984

ARIADNE: A Knowledge-Based InteractiveSystem for Planning and Decision Support

ANDREW P. SAGE, FELLOW, IEEE, AND CHELSEA C. WHITE, III, MEMBER, IEEE

35

Abstract- The development of an interactive planning and decisionsupport process for multiple criteria alternative selection situations isdiscussed. Probabilities, utility scores for the lowest level attributes, andattribute trade-off weights, i.e., the parameters, can be imprecisely de­scribed by set inclusion. Within a specified structural model of the decisionsituation, the process allows the decisionmaker to iteratively select the mixof parameter value precision and alternative ranking specificity. By select­ing this mix, the decisionmaker is able to direct the alternative selectionprocess in an interactive manner, using alternative selection strategiesbased on behaviorally meaningful dominance search strategies. Emphasis isplaced on the motivation of the research and the behavioral relevance ofthe support process. References in the bibliography provide further analyti­cal and behavioral discussions related to this process.

1. INTRODUCTION AND MOTIVATION FOR THE

RESEARCH

I T HAS BEEN observed that the process of choosingamong multiattributed alternatives often involves an ini­

tial search for a dominance structure and ultimate identifi­cation of a set of nondominated alternatives, alternativeswhich are not worse than any other alternative on anyattribute and which are better than each other alternativeon at least one attribute. In most decision situationshowever, no single alternative dominates all other alterna­tives, at least initially. In such decision situations, thedecisionmaker typically" adjusts" the structure of the deci­sion situation and parameter values within this structure soas to identify a dominance structure which contains asingle nondominant alternative. This search may involverational activities, such as aggregation of attributes andcompensatory trade-offs through determination of judg­mental weights. Alternatively, it may involve various ruleswhich may be quite flawed. Examples of such rules are 1)lexicographic ordering, in which the best alternative on themost important attribute is selected, and 2) sequentialpairwise comparison of alternatives using a preferencerelation that is a function of the two alternatives beingcompared. In this latter case, nontransitive preferencesmay easily result due to the fact that the contextual rela­tion used to determine preferences changes from binarycomparison to binary comparison.

A variety of holistic, heuristic, and wholistic judgmentalactivities will typically be involved in the search for a

Manuscript received February 1983; revised August 1983. This workwas supported by the Office of Naval Research under Contract N00014­80-C-0542.. The a~th?rs are with the Department of Systems Engineering, Univer­

sity of Virginia, Charlottesville, VA 22901.

dominance structure among the alternatives. These take onvarious forms and mixtures of formal knowledge-based,rule-based, or skill-based activities as deemed appropriatefor the task at hand [1], [2]. Especially with a large numberof alternative courses of action under consideration, thedecision process will typically involve mixed scanning,where some noncompensatory rule is first used to eliminategrossly inappropriate alternatives. This is then followed by~ne or more compensatory information evaluation opera­nons that results in a dominance structure which enablesfinal judgment and alternative selection.

The research discussed here is based upon the hypothesisthat people are able to evaluate alternative plans anddecisions efficiently and effectively and with low stresswhen a clear dominance pattern exists among alternativesthat allows the establishment of a sufficiently discrimina­tory priority structure. Our goal is to provide aknowledge-based decision support process that enhancesthe quality of the dominance structure used for judgmentand choice.

Often people process information poorly through variousforms of selective perception. A typical flaw involves ignor­ing potentially disconfirming information in order to per­ceive a dominance pattern among alternatives when nosuch pattern exists. Another flaw is to evaluate one non­dominant alternative incrementally higher than anotherone after the introduction of alternatives asymmetricallydominated by the first nondominant alternative but not bythe second. Sequential pairwise comparison of alternativesoften assumes an implicit contextual relation », whereA ~ B suggests that choosing A and rejecting B is prefer­able to choosing B and rejecting A [3].

Thus the preference relation is alternative dependent ingeneral, and non transitive results can be expected from itsuse in unaided situations. Agenda-dependent results willtypically occur in aided situations if we force transitivitythrough the use of transitive inference and associated ne­glect of questions that would have provided results whichwould have disconfirmed the transitivity assumption. Thusthere seems much motivation to provide assistance in thissearch for a dominance structure that will assist in theprocess of judgment, choice, and decision [4], [5].

In this paper, we provide an overview of our research tothese ends. The next section will present a summary of thefeatures and structural constructs of our decision supportsystem. The following section presents a more detaileddiscussion of these structural constructs and introduces

0018-9472/84/0100-0035$01.00 ©1984 IEEE

36 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. SMc-14, No.1, JANUARY/FEBRUARY 1984

some of the modes in which the support process can beused. Then we discuss some behavioral issues that relate tothe conceptual design of ARIADNE. Next we present abrief description of some of the algorithmic constructs thatallow and assist the development of dominance structuresfor the various modes in which use of the support system ispossible. Some very simple illustrative examples are fol­lowed by a section devoted to conclusions and extensionsto our research that are currently in progress.

II. FEATURES OF THE DECISION SUPPORT SYSTEM

We now investigate concepts for the design and eval­uation of an interactive knowledge-based planning anddecision support system which combines, or allows thecombination of, several evaluation rules and contingencystructures often used as a basis for evaluation, prioritiza­tion, judgment, and choice. We have developed a knowl­edge-based system to interactively aid planning and deci­sion support processes through encouragement of the searchfor a dominance structure that is behaviorally realistic andrational, from both a substantive and procedural view­point. The support system is called Alternative RankingInteractive Aid based on DomiNance structural informa­tion Elicitation (ARIADNE). The support system allowsthe use of various integrated forms of wholistic, heuristic,and holistic reasoning in an aided search for dominanceinformation among identified alternatives. We believe it tobe flexible enough to match diverse decision situations andenvironments closely in order to support varying cognitiveskills and decision styles, thereby enabling planners anddecisionmakers to adapt its use to their own cognitiveskills, decision styles, and knowledge.

Our efforts have concerned choice-making situations un­der certainty and under risk, primarily for the single deci­sion node case. This formulation allows the considerationof a variety of imprecisely known parameters such asattribute trade-off weights, outcome state values on lowestlevel attributes, event outcome probabilities, and variouscombinations of these. Parameter needs are determinedfrom the structure of the decision situation, as elicited fromthe decisionmaker during the formulation and analysissteps of the decision support process. We consider theseformulation and analysis steps to be outside the scope ofour present software developments but recognize the essen­tial need for them in a complete decision support process.

The decision situation structural model may representdecisions under risk or under certainty. The attribute treerepresenting the features of decision outcome states may bestructured and/or parameterized in a top-down or bottom­up fashion through use of ARIADNE. A single-level struc­ture or a multiple-level hierarchical structure of attributesmay be used with the choice of these being at the discretionof the decisionmaker. Multiple decision node situationsmay be approached through a goal directed decision struc­turing approach in which the growth of the structure ofalternative decisions and event outcomes is guided bysensitivity-like computations obtained through use of theARIADNE algorithms [6]-[8].

Parameters are elicited from the decisionmaker in theform of equalities and inequality bounds. A variety ofmathematical programming approaches and graph theoryhave been used to generate interactive displays of prefer­ence digraphs. These mathematical programming ap­proaches are used to determine dominance structures foralternative prioritization that are based on parameter infor­mation elicited from the decisionmaker. At present, only alinear programming approach will yield necessary andsufficient conditions for determination of a priority struc­ture and computational times that are consistent withinteractive decision aiding. This requires that we elicitstructural parameter information in a slightly restrictedform which we denote the "behaviorally consistent infor­mation set" (BCIS). Often this BCIS will be in such a formthat solution of the generally nonlinear programming prob­lems associated with determination of dominance struc­tures can be replaced by the solution of simple computa­tionally amenable linear programs with bounded variables.The major simplification associated with eliciting parame­ter imprecision in a prespecified structural format, how­ever, is in the natural language dialogue needed to establisha model of the decision situation.

The purpose of the graph theory algorithm is to allowthe construction of a domination digraph or dominancesmrctural model. This digraph is a pictorial representationof the ordinal preferences as determined from a dominancereachability matrix. This matrix is determined by the linearprogramming algorithms from the decision situation struct­ural model and parameters elicited from the decision­maker. These domination digraphs encourage either selec­tion of a preferred alternative, or further iteration using theaggregated preference information for feedback learning.

An inverse aiding feature is currently being incorporatedinto the decision support system. This feature allows thedecisionmaker to make wholistic skill-based prioritizationsamong alternatives. These prioritizations may be acrosssome or all identified alternatives, at the top level of thehierarchy of attributes or at some intermediate level. If weelicit numerical bounds on the attribute scores for thoseattributes which are subordinate to and included within theattribute at which alternatives are prioritized, then boundson attribute weights, consistent with the wholistic prioriti­zation, may be determined by using a linear programmingapproach. Alternately, if weights are specified, then it ispossible to determine bounds on alternative scores on thoseattributes subordinate to the attribute at which prioritiza­tion was made through the use of linear programmingalgorithms.

As alternatives are identified and prioritized, updates onthese bounds are made available. The results obtained fromusing the inverse aiding feature are, in may ways, compara­ble to those obtained from the regression analysis basedsocial judgment theory [9]. This approach provides weightidentification only, with a "confidence" measurement con­cerning the validity of weights; cardinal preferences areassumed. Results in the form of bounds on, or ranges of,weights are available with a very few alternative prioritiza-

SAGE AND WHITE: KNOWLEDGE-BASED INTERACTIVE SYSTEM

tions in the inverse aiding approach. The prioritizationsneeded may involve a mixture of cardinal and ordinalpreferences. For a large number of prioritizations, theinverse aiding approach may become cumbersome compu­tationally compared to the regression-based approach,where additional information may be easily processed in asequential fashion.

The combination of inverse and direct aiding to enhancedecisionmaker specification of imprecise values, weights,and probabilities enhances the usefulness of ARIADNEsince it allows for judgments and their explanation, using acombination of formal knowledge-based and skill-basedmodes. This enhanced usefulness will also occur throughencouragement to the decisionmaker to become more awareof relevant alternative courses of action and to identifynew alternatives on the basis of feedback learning of theimpacts of alternatives upon issues and objectives in abehaviorally relevant way that, hopefully, encourages"double-loop learning" [10].

III. STRUCTURE OF ARIADNE

A complete set of activities envisioned in using thesingle-stage, or single decision node, version of ARIADNEinvolves the following set of activities.

A. Formulation of the Decision Situation

1) Define the problem or issue that requires planningand decisionmaking by identification of its elements interms of a) needs, and b) constraints or bounds on theissue.

2) Identify a value system with which to evaluate alter­native courses of action, and identify objectives or attri­butes of the outcomes of possible decisions or alternativecourses of action.

3) Identify possible alternative courses of action or op­tion generation.

B. Analysis of the Decision Situation

1) Determine outcome scenarios.2) Identify decision structural model elements, that is,

those elements or factors from the conceptual formulationframework which appear pertinent for incorporation into adecision situation structural model.

3) Structure decision model elements:

a) structure decision tree,b) structure information acquisition and processing

tree-which may be part of the basic decision tree,and

c) structure attribute tree or objectives hierarchy.

4) Determine independence conditions among elementsof the attribute tree and decision alternatives.

5) Identify potential for the use of deficient informationprocessing heuristics, and provide appropriate debiasingprocedures.

37

6) Determine impacts of, or outcomes that may resultfrom, alternative courses of action.

7) Encode uncertainty elements in the form of eventoutcome probabilities, or bounds on these, to the extentpossible.

8) Identify risk aversion coefficients, if needed, to theextent possible.

9) Identify preference or value functions, or bounds onthese functions, to the extent possible.

10) Identify attribute weights, or bounds on these func­tions, to the extent possible.

11) Identify wholistic preferences among alternatives tothe extent that this is possible.

12) Identify possible disjunctive and conjuctive aspects,or thresholds for attributes, of identified alternative coursesof action.

C. Evaluation and Interpretation of the Outcome ofAlternative Courses of Action

1) Identify a decision aiding protocol or plan for evalua­tion and interpretation of the decision situation.

2) Identify potential for use of deficient judgment heur­istics.

3) Use conjunctive and/or disjunctive scanning toeliminate very deficient alternatives and retain alternativesmeeting minimum acceptability criteria across attributes.

4) Determine the maximum amount of domination in­formation possible.

a) Display domination digraph.b) Identify alternative courses of action which could

not be among the N most preferred alternatives. Normally,these are deleted from further consideration.

c) If the decisionmaker can select an alternative forimplementation by wholistic judgment, or prioritize theremaining alternative set through heuristic elimination, thengo to step 6) of evaluation and interpretation (SectionIII-C).

d) If a choice cannot be made, then assess furtherinformation about values of imprecisely known parametersby iterating through steps 6)-11) of analysis (Section III-B),then return to step 1) of the evaluation and interpretation(Section III -C). Many possibilities exist for obtaininggreater alternative evaluation specificity such as

i) setting higher aspiration levels or aspects,ii) moving up the attribute tree by determination of

a subset of attribute tradeoff weights,iii) "tightening" bounds on attribute trade-off

weights,iv) tightening bounds on ev-ent outcome probabili­

ties, possibly through information processing up­dates,

v) tightening bounds on values or preference func­tions.

5) If the decisionmaker has provided (partial) wholisticpreferences as part of the analysis effort, use these with the

38 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. SMc-14, No.1, JANUARY/FEBRUARY 1984

inverse aiding feature of the aid to determine bounds onattribute weights implied by these preferences such as toprovide learning feedback to decisionmaker.

6) Conduct sensitivity analysis. Provide the decision­maker with an indication of how sensitive the optimalaction alternative, or prioritization of alternatives, is withrespect to changes in values and information about im­pacts.

7) Evaluate validity and veracity of the approach. En­courage judgment concerning whether the formulation,analysis, and interpretation are sound. If not, encourageappropriate modification to structure and parameters asso­ciated with the decision situation, including identificationof additional attributes and alternative courses of action.Then, iterate back to an appropriate step and continue.

In our work to date, we assume that the details of issueformulation and analysis are accomplished external to theinteractive aid itself. A variety of procedures exists foraccomplishing these tasks [11]. Our research assumes that,an issue formulation structure exists and that the impactsof alternatives are known. These are provided throughvarious elicitation activities. We do not envision that thesoftware we develop for interactive interpretation, includingevaluation and prioritization, will generally be suitable foruse independent of a trained decision analyst. Whethersoftware can be evolved to result in an appropriate "standalone" aid is very dependent upon the environment andother factors that constitute the contingency task structurefor a specific situation. In situations which are repetitiveand environments which are stable, such as in health careor equipment fault diagnosis situations, it seems entirelypossible to design useful "stand alone" aids. In moststrategic, and in many tactical situations there will not be astable underlying structure that will easily allow this. Theactivities involved in issue framing and the identification ofa dominance structure appropriate for decisionmaking areoften very situation dependent.

A number of considerations influence planning and deci­sion support processes. The person using a decision sup­port system should be aware of these considerations if bestuse of the aiding process is to be obtained. Generally, theseconsiderations involve the operational environment and thefamiliarity of the decisionmaker with the environment andtask at hand. It is the interaction of these factors thatinfluence:

1) behavioral characteristics of the decisionmaker,2) interaction between decisionmaker and analyst,3) choice of computer-based support for decisionmaker

analyst interaction

Among the behavioral characteristics of the decision­maker that influence aiding consideration strongly are thefacts that the decisionmaker

1) is often impatient with time consuming and stressfulassessment procedures;

2) wants to see some preliminary results promptly ifthese are needed or wanted;

3) may lack interest in interacting directly with complexquantitative procedures for decision aiding that donot seem tailored to the specific contingency taskstructure of the issue at hand; and, as a consequence,

4) requires a decision aiding approach that adapts to thedecisionmaking style appropriate for the decision­maker in the given contingency task structure.

A number of considerations influence the most desirableinteraction between the decisionmaker and the analyst. Theinteraction must be such that these result:

• a list of objectives and an objectives hierarchy,• a list of alternatives, and• a list of outcomes for each alternative.

The extent of the need for the use of these identified listswill vary greatly with the "expertise" of the decisionmaker.A major task of the analyst in the formulation and analysisportion of the aiding effort is to assist the decisionmaker inobtaining these lists in a behaviorally relevant and realisticmanner. The analyst must also ensure, to the extent possi­ble, that

1) the foregoing lists are reasonably complete;2) the lowest level objectives are additively independent;3) the alternatives are mutually exclusive; and4) the outcomes that follow from each alternative are

mutually exclusive and exhaustive.

The nature of the interactive process is such that iterativechanges can be made in terms of addition or deletion ofalternatives and attributes. Nevertheless, there are signifi­cant advantages in attempting to be reasonably complete atthe start of the interpretation portion of the process.

The decisionmaker must provide the analyst, followingbehaviorally realistic elicitation procedures, with informa­tion regarding 1) alternative scores on lowest level attri­butes, 2) trade-off weights, 3) probabilities, and 4) relativerisk aversion coefficients, or else appropriate ratios orbounds on these quantities which represent the precisionthat the decisionmaker believes appropriate or is capableof providing for the given decision situation.

Many computer-based support perspectives evolve fromdecisionmaker-analyst interaction considerations. A goalof all decision support system design efforts is to obtain"friendly" software, software that is friendly both to thedecisionmaker and the analyst. In particular, the analystmust be able to interpret the decisionmaker's structuraland parameter information for input to the computer. Todo this may require 1) redefining the outcome space, suchas redefinition of attributes to ensure satisfaction of inde­pendence considerations, and 2) describing parameter in­formation in terms of inequalities (or more generally setmembership).

The analyst must be able to interpret computer output ina fashion that facilitates the decisionmaker's understanding

SAGE AND WHITE: KNOWLEDGE-BASED INTERACTIVE SYSTEM

and decisionmaking abilities. The analyst must be able toassist the decisionmaker in responding to the followingquestion that is central in our interactive knowledge-basedsupport system: Has sufficient preference and structuralinformation been elicited from and provided to the decision­maker for alternative selection, or is more information re­quired for identification of a dominance structure that isrelevant and appropriate for quality decision support? Changereceptivity must therefore be an inherent part of this userfriendliness.

If the decisionmaker feels that an alternative can beselected for action implementation at any stage in theinteractive aiding effort, the analyst must be able to en­courage decisionmaker judgment concerning whether ornot the issue formulation, analysis, and interpretation aresound. If the issue formulation, analysis, and/or interpre­tation are not perceived as sound by the decisionmaker, theanalyst must be able to encourage appropriate structuraland parameter value modification, typically by means ofsensitivity analysis, in order to insure effective, explicable,and valid planning and decision support. If the decision­maker cannot choose an alternative from among thoseconsidered, the analyst must be capable of eliciting furtherstructural and/or parameter information to enhance ap­propriate selection of alternative courses of action.

One very important feature of a knowledge-based systemfor planning and decision support is encouragement to thedecisionmaker for generating new options, outcomes, andattributes at essentially any point in the aiding effort andability to evaluate these new options properly. The analystmust be able to cope with this additional informationunder the assumption that

1) the new information is consistent with previouslyobtained information, or

2) the new information is not consistent with previouslyobtained information due to a) structural inconsistencies orb) parameter inconsistencies.

Thus the capacity to resolve potential inconsistenciesthrough interaction with the decisionmaker must existwithin the planning and decision support process. Theindirect or inverse decision aiding feature should be ofparticular value to this end. In a "policy capture" likefashion, this indirect feature will allow identification ofbounds on attribute weights in terms of wholistic prefer­ences among some, or all, alternatives. In the direct aidingfeature, values, weights, and probabilities are identifiedand prioritization of alternatives result from this. Com­bined use of the direct aiding feature with indirect aidingshould result in much Iearning feedback concerning rela­tions among the various modes of judgment.

ARIADNE, as we have noted, does not contain softwareto assist in the formulation and analysis portion of theplanning and decision support effort. It is in these twosteps that alternative choices, attributes, and decision im­pacts or 'outcomes are elicited or identified. Our effort ismuch more concerned with the interpretation part of adecisionmaking effort, that is to say, how information

39

is processed concerning formulation and analysis-basedquantities such as probabilities, values, weights, ratios, andbounds upon these. Weare concerned also with the way inwhich this information is aggregated, by any of a variety offormal knowledge-, rule-, or skill-based modes of cognitionthat result in judgment and choice. We recognize thedifficulties in separating the tasks of formulation and anal­ysis from those of interpretation. Difficulties exist at thesystems management level since the way in which peoplecognize a problem, as part of the contingency task struc­ture of a particular situation, determines the way in whichthey will go about resolving it. Thus the performanceobjectives, information processing style, and decision stylethat are most appropriate and likely to be used for a giventask are very much dependent upon the task itself. When aparticular concrete operational or skill-based strategy hasyielded previous satisfactory results, many people will tendto use that strategy unquestioningly and uncritically in newsituations perceived to be similar. This can result in veryunsatisfactory judgments and choices in decision situationsthat have changed and that are not recognized as differentfrom familiar past situations. This may result in prematurecessation of search and evaluation of alternatives prior toidentification of quality strategies, even for familiar situa­tions. The efforts can be devastating in unfamiliar environ­ments that are not so recognized [12].

The strategies which a decisionmaker will desire to usefor interactive interpretation will be strongly dependentupon the way in which the task requirements are initiallycognized. This will influence the objectives, attributes, andalternatives generated in the formulation step and the valuescores or impacts associated with them in the analysis step.The input information to the interpretation step is just thisinformation. Adequacy of the interactive interpretationstep will clearly by dependent upon the "quality" of theinformation input to it.

Many recent studies [13] have indicated that peopleoften construct selectively perceived simple deterministicrepresentations of decision situations that make informa­tion processing easy and which do not reflect the complexi­ties and uncertainties that are associated with the actualsituation. A goal of a decision support system is to encour­age wide scope perceptions and associated informationprocessing. The process used to assess probabilities, utili­ties, and weights will doubtlessly affect the quantities thatare elicited. It is possible, for example, that a poor elici­tation procedure may, unknowingly or knowingly, createrather than measure values [14]. An advantage to formalsupport for planning and decisionmaking processes is thatit is possible to conduct a search for inconsistent judgmentand perhaps even detect flawed information processingheuristics if process tracing is used. When inconsistenciesare discovered, it then becomes possible, at least in princi­ple, to examine the judgment process to determine whichjudgments imply flawed information processing, and/orincoherent or labile values, and/or deficient decision rules.A major ultimate goal, outside the scope of our present

40 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. SMc-14, No.1, JANUARY/FEBRUARY 1984

study, is to suggest debiasing and other corrective proce­dures to enhance the quality of human informationprocessing .and decision rule selection.

This mixed scanning based planning and decision sup­port system is based upon rational search for a dominancestructure which will enable exposure of some of theprocesses upon which' judgment and choice is based. Inparticular, it enables determination of the precise point in~ dominance structure search process when a decision­maker is able to select a single nondominated alternative.Thus we should be able, for example, to detect violation ofthe regularity and similarity, hypotheses that often occurwhen a number of asymmetrically dominated alternativesexists [15]. More importantly, we should be able to correctfor this without resorting to a complete elicitation ofprecise parameter information and prioritization of allalternatives. This activity is often stressful and time con­suming, may require perspectives outside the experientialfamiliarity of the decisionmaker, and allows few resultsuntil conclusion of the aiding effort.

The overall process described here appears well suited toaccommodating the fact that neither individuals nor groupspossess static decision styles. capable of being stereotypedand captured by an inflexible support process. It is specifi­cally recognized that an interactive process is needed thatis capable of adaptation to a variety of decision styles thatare contingency. task structure dependent. System designshould reflect the realization that it is generally not possi­ble . to define a problem or issue fully until one knowspotential solutions to the issue. A major cause of this is thefact that information to define the issue fully generallybecomes available only as one evaluates potential solu­tions. Planning and decisionmaking will therefore neces­sarily be iterative.

IV. BEHAVIORAL RELEVANCE

Our decision support system design paradigm is basedupon a process model of decisionmaking in which a personperceives an issue which may require a change in theexisting course of action. On the basis of a framing of thedecision situation, one or more alternative courses of ac­tion, in addition to the present option which may becontinued, are identified. A preliminary screening of thealternatives,using conjunctive and disjunctive scanning,may eliminate all, but one alternative course of action.Unconflicted adherence to the present course of action orunconflicted change to a .new option may well be themetastrategy for judgment and choice that is adopted if thedecisionmaker perceives that the decision situation is afamiliar one and that the stakes are not so high that a morethorough search and deliberation is needed [12].

Alternately, if the decision environment is an unfamiliarone or the stakes associated with judgment and choice arehigh, more vigilant forms of informative acquisition, analy­sis, and interpretation are called for. This desire for m~re

vigilant information processing leads to a search for a

dominance pattern among alternatives, the search for newalternatives that are not dominated by presently identifiedalternatives, and the elimination from further considerationof dominated alternatives. If no single nondominated alter­native is found, adjustments to the dominance structure ofalternatives are made through various forms of cognitiveactivity such as attribute aggregation, additional informa­tion acquisition and analysis, and identification of addi­tional attributes and!or alternatives. This is continueduntil the structure of needs, objectives, attributes, andalternative action options and their impacts is such thatidentification of a single nondominated alternative results.This single,alternative may well represent a combination ofsubalternatives. If the time and experience to accomplishthese cognitive activities is insufficient, hypervigilance gen­erally results. The decisionmaker is then in a situationwhere the present course of action is diagnosed as unfor­tunate, and there is not enough time and experience' toallow identification and evaluation of an appropriate alter­native course.

Given sufficient time and experience, vigilant informa­tion processing often results from the aforementioned tasks.Fig. 1 presents some salient features of this dominanceprocess model for search, discovery, judgment, and choice.

The proper mode of judgment and choice depends uponthe decisionmaker's situation diagnosis of the contingencytask structure. Here, "proper" decision behavior is basedupon the assumption. that the environment, the task, theexperiential familiarity with the task, and the environmentthat constitutes the contingency task structure are di­agnosed correctly. If this is not the case, then the strategiesleading to unconflicted change, adherence, or vigilant in­formation processing may be significantly flawed. The roleof the contingency task structure in situation diagnosis andin influencing, at a meta or systems management level, theprocess of judgment and choice is seen to be a veryimportant one. It leads to a four-element representation ofsituation diagnosis as shown in Table I which also presentsa typology of the decisionmaker whose behavior may re­flect in judgment activities in either of the four quadrantsof this figure. In the upper right quadrant, where truemastery or grand mastery of a decision situation results, itis doubtful that any decision support system will be ofmuch direct and personal use to the decisionmaker. Never­theless it may have much indirect use in enabling acquisi­tion of a knowledge base and as a useful pedagogical orlearning system for others.

Many realistic paradigms have been made ,of the processof judgment and choice. We believe that the dominanceprocess model described here is not inconsistent with theprimary features and intensions of these descriptive models.Our purpose, however, is to develop a conceptual designfor a prescriptiveapproach to judgment and choice that willaid in the search for better decisions. We recognize that atruly rational approach to prescriptive decisionmaking mustbe cognizant of the process of decisionmaking as it evolvesin a descriptive fashion, that is to say, process rationality,

SAGE AND WHITE: KNOWLEDGE-BASED INTERACTIVE SYSTEM 41

Potential Need forChange Recognized

Formulation of DecisionSituation (including identification

of alternative options)

Non CompensatoryScanning

Initial Choice of aSingle Alternative

Examine Single Alternative foriolation of "Significant· Dominance"

or Aspect Requirement

No ViolationDetected

ViolationDetected

Is there Sufficient Time andExperiential Familiarity to

Identify Better Courses of Action

Dominance Structuring

Identify Broad Scope Rangeof Alternatives

YesNo

Use Wholistic, Heuristic orHolistic Judgment in Attempt toSelect Nondominant Alternatives

Selectionr---------~Pos sib1e

SelectionNot Possible

Select andImplementAlternative

Can AdditionalAlternatives be

Identified

No

Yes

Fig. 1. Descriptive dominance structural model of decision process.

42 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. SMc-14, No.1, JANUARY/FEBRUARY 1984

TABLE ITAXONOMY OF REACTIONS TO DECISION SITUATION IN TERMS OF

PERCEIVED VERSUS ACTUAL KNOWLEDGE

or it will not be possible to evolve substantively rationalsupport systems.

It is important that an appropriate decision supportsystem be capable of assisting the decisionmaker throughencouragement of full information acquisition, includingthat which may disconfirm strongly held beliefs, and in theanalysis and interpretation of this information such as toavoid a variety of cognitive biases and poor informationprocessing heuristics that may lead to flawed judgment andchoice [2], [13].

A realistic decision support process is necessarily itera­tive. Several desiderata follow from this.

1) We should allow for top-down or bottom-up structur­ing of the attributes of outcomes, or impacts of decisions.The "true" or "hierarchy" of attributes should be struc­tured to the depth believed appropriate by the decision­maker.

2) Rather than force a decision situation structural modelin the form of'a tree, we should encourage the decision­maker to identify a cognitive map of goals, objectives,needs, attributes, alternatives, and impacts that reflects theway in which he perceives diagnostic and causal inferencesto occur. At some later time this cognitive map may beused to structure a multinode decision tree which repre­sents substantive rationality, but not necessarily processrationality.

3) We should encourage identification of alternativecourses of action, additional attributes of decision out­comes, and revisions to previously obtained elicitations, atany point in the decision support process as awareness ofthe decision situation and its structure grows through useof the support system.

4) We should not force a person to quantify parametersto the extent that this becomes overly stressful or behavior­ally and physically irrelevant in view of the inherent uncer­tainty or imprecision associated with the knowledge ofparameters characterizing the decision situation structuralmodel or their assessment.

These have two primary implications with respect to ourinterpretation efforts. We allow for revision in the elicitedstructure of the decision situation and for the identificationof new options as awareness of the decision situationgrows. Also, we do not require the decisionmaker to quan-

Perceived Knowledge Level

A. The Attribute Tree and Decisions Under Certainty

It is possible to use either a hierarchical tree structure ora single-level structure of attributes, each of which areshown in Fig. 2. Fortunately, the relations between theweights associated with the tree structure and the single­level structure are easily determined. They are given in Fig.3(c). Linear inequalities in terms of hierarchical weights orweight ratios w/ become linear inequalities in terms ofsingle-level weights Pi' It is this fact that allows us to usethe single-level representation in the ARIADNE software

tify parameters beyond the level felt appropriate for thesituation at hand. If the decisionmaker feels comfortable inexercising precision with respect to factual outcomes, thisis perfectly acceptable and desirable, but parameter impre­cision should be allowed if we are to have a realisticsupport process.

ARIADNE allows parameter imprecision in order tosatisfy this quantification relevancy requirement, as doapproaches based on fuzzy set theory [16]. We encouragethe decisionmaker to specify precise values or numericalranges for facts and values. Thus we allow, for example,expressions for alternative (a) scores on attributes (i) inthe form 0.2 ~ vi(a) ~ 0.5, weights associated with attri­bute (i) in the form 0.2 ~ Wi ~ 0.4, and probabilities ofevent (i) resulting from alternative (a) in the form 0.3 ~

PiCA) ~ 0.45. We allow ordinal representations in the lin­ear forms vi(a) ~ vi(b) ~ viCe), 2Wi < Wj < wk ' Ij(a) ~PiCa) ~ 3Pk(a), or in similar forms. Quantification of im­precision in the form of numerical bounds on parametersalways leads to behaviorally consistent information sets(BCIS). Sometimes totally ordinal information may needfurther quantification in order to make the precision andrigidity of the mathematics correspond to the intensions ofthe decisionmaker in making a purely ordinal specification.This is generally not needed to obtain solutions but ratherto obtain parametric models that are faithful to the under­standings of the decisionmaker. For example, that ordinalalternative score inequalities 0 ~ vi(a) ~ vi(b) ~ viCe) ~ 1are satisfied by the relations 0 ~ vi(a) ~ 1 - 2t, W ~ vi(b)~ 1 - t, 2W ~ viCe) ~ 1 for small positive t and Wwhichin the limit become zero. It will generally not be the casethat the decisionmaker would express this much impreci­sion and would wish to see it more fully quantified toreflect (subjective) beliefs. It is, therefore, important that asimple and informative display of value scores, weights,and probabilities be provided to the decisionmaker. Thiswill enhance interactive use of the support system and willenable learning of the impact of these parameters, andassociated imprecision, upon decisions.

v. ALGORITHMIC CONTENT

This section will discuss some of the algorithmic contentsupporting the decision support system. To facilitate read­ing, we will make each subsection of this description moreor less independent of other subsections.

MASTER

Wholistic intuitive judgmentwill likely lead to highquality decision throughunconflicted change orunconflicted adherence

Decisionmaker may be amaster of the art ofself deception

NOVICE

Decisionmaker is unawarethat considerable judgmentability exists

Decisionmaker is awareof need for decisionsupport to enable holis­tic judgment

SAGE AND WHITE: KNOWLEDGE-BASED INTERACTIVE SYSTEM

~ ~ ~ ~P3 P4

Vl(X i ) V2(X i ) V3(X i ) V4(X i ) vs(x i)

(a)

43

(b)

Pj =product of weights in path from vj(xi) to v(x i)

sum of p. weights over those i .. which are sUbordinate to attribute a~J - -----:~-----:--__:__~__=__--'--:-----::_:__-

wk - sum of p. weights over those i which are subordinateto the attribute to which attribute a~ is subordinate

Examples P2 = w~ wI wf

w~=~Pl+P2+P3

w~ =P4 + Ps + Pn

(c)

Fig. 2. Comparisons between single-level and hierarchical attribute structures. (a) Single-level attribute structure. (b)Hierarchy of attributes. This structure may be equivalent to that of (a). (c) Relationship between weights.

and still to make assessments in terms of the hierarchicalweights w/ at any level of the hierarchy.

For the case of decisionmaking under outcome certainty,we know that the ith outcome Xi follows from the ithalternative. We therefore have for the value of the ithalternative, vk(a;) the expression

N

v (ai) = L pkVk ( a;) . (1)k=l

We say that alternative i has a higher value score thanalternative} when

N

aV;jmin = minavij = min L Pk[vk(a;) - vk(aj)]A A k=l

= min pT[ V ( a;) - v ( ai )] ~ 0 (2)

A

where A denotes the set of imprecise parameters overwhich the extremization is conducted. This set is restrictedthrough the eliciting process such that A E A. p and v arevectors with components Pk and Vk. For the realistic casewhere weights and attribute scores for each attribute arefunctionally independent, (2) becomes

N

aVijmin = minpTV(i, }) = L PkVk(i, }) ~ 0 (3)P k=l

where

V(i , }) = min. [ v ( a;) - v ( aj ) ] ( 4)v(aj). v(aj)

IS a vector whose components are

min [vk(a;)-vk(aj)], k=1,2,···,N.vk(ai)vk(aj)

We emphasize again that the vector minimization to de­termine V(i, }) is meaningful only when vk(a;) is function­ally independent of vm(a;) and vm(aj) for k =F m. In manycases, the vk ( a;) are elicited so as to be functionallyindependent of the vk(aj ) , and then we have

V(i,}) = minv(a;) - maxv(aj) = .f(a;) - V(aj)v(ai) u(aj) .

(5)

where .f and V denote the minimum and maximum valuesof the value score vector on the specified alternative.Determination of the solution to (3) requires solution of alinear program (LP) for each ij pair. If A alternatives exist,we will need to solve no more than A(A - 1) LP's toresolve (3). We may need to solve fewer LP's than thissince if dV;jmin > °we can assume that aVjimin < 0 withoutsolving the associated LP and know that a; >- aj • Solutionof (4) for a specific i and} will involve a single LP. Thus wehave A(A - 1) LP's to solve to determine V(i, }) for all iand}. Generally, these linear programs are extraordinarilysimple to solve and result in necessary and sufficientconditions for a; >- aj •

As one simple example, let us consider the alternativescore matrix on lowest level attributes as seen in Table II.We assume that the decisionmaker specified a single-levelattribute tree and is able to estimate the weights 0.1 ~ PI~ 0.2,0.2 ~ P2 ~ 0.4, and 0.3 ~ P3 ~ 0.7. Of course, theseweights must sum to one, and so we have PI + P2 + P3 = 1.We have already specified utilities in the max-min form[V,.fl; nothing more is needed here. To see whether a >- b,we need to see whether (3) is satisfied. Thus we

44 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. SMc-14, No.1, JANUARY/FEBRUARY 1984

TABLE IIVALUE SCORE ON ATIRIBUTES

Alternative

abcd

Attribute 1max score min score

o 01 10.5 0.30.4 0.2

Attribute 2max score min score

1 1o 00.4 0.20.18 0.1

Attribute 3max score min score

o 00.8 0.61 10.7 0.5

TABLE III

Attribute wiAlternative max score min score

Attribute 3 (or w;)max score min score

Fig. 3. Preference structure for simple example. abcd

0.80.40.440.268

0.60.20.240.14

o0.810.7

o0.610.5

utility

Fig. 4. Hierarchical tree of attributes and associated weights.

This is an LP with bounded variables, a particularly simpleform of linear program. We assign maximum weights tothe most negative Ea - Vb components until we are all outof weight. The weights are found by setting all weights attheir minimum value and then allocating additional weightwhere it will do the most good. So we use PI = 0.2,P3 = 0.6, P2 = 0.2 and getaVabmin = -1(0.2) + 1(0..2) ­0.8(0.6) = - 0.48, which is not greater than zero. Thus it isnot possible to have a >- b. To see if b >- a we examine

We use P2 = 0.4, P3 = 0.5, PI = 0.1 and get aVbamin = O.So we conclude that b >- a. We determine that c >- bandb >- d, using just this procedure, such that we have thepreference structure of Fig. 3. Thus c is the preferredalternative here.

It may well be, however, that the decisionmaker (DM)visualizes the attributes in a hierarchical form as shown inFig. 4. The same alternative score matrix used earlier is stillappropriate here. If the DM feels comfortable in evaluatingweights associated with attributes 1 and 2 (PI and P2) butnot with 3 (P3)' then a multilevel hierarchy of attributesmay be assumed. We could attempt to assist the DM byaggregating up the attribute tree. Alternately, we coulddetermine whether or not the relationship between attri­butes 1 and 2 is sufficient to establish a single non-

dominated alternative by converting to the single-levelweight form.

We will illustrate calculations using the first approach.Suppose that the DM says wi ~ w}. Then the analystmight use this ordinal expression or might convert to acardinal representation and say that at level 1 the weightsare such that 0 ~ wi ~ 0.5, 0.5 < w} < 1.0. Following arequest to be more explicit, perhaps the decisionmakerindicates that 0.2 ~ wi ~ 0.4 and 0.6 ~ wi ~ 0.8. We nowaggregate attributes 1 and 2. Based on this information, wecan calculate a maximum and minimum score for theutilities of the alternatives on the second level attribute wf.We obtain the aggregated value score matrix shown inTable III. No domination pattern exists at all, so we mustgo further.

As often occurs in problems of this sort, the level 2alternative scores are not in proper 0-1 range. If the DMfeels more comfortable in seeing these scaled over a 0-1range, this can easily be done. Otherwise, the DM is askedto consider the difference in scores from max to min onattribute wf [0.8 on alternative a and 0.14 on alternative d]and express the importance weight of the difference onattribute 3 of the difference between the maximum andminimum scores on alternatives c and a. Suppose thatinequalities of the form 0.2 ~ wf ~ 0.35 and 0.65 ~ wi ~0.8 finally result. We can then determine a table of maxi­mum and minimum alternative scores:

Alternative Maximum Score Minimum Score

a 0.28 0.12b 0.72 0.46c 0.888 0.734d 0.6136 0.374

Thus we have the preference digraph of Fig. 5 which isslightly different from that obtained earlier. The conclusionis the same, however. Alternative c is the best alternative.This particular approach used to aggregate up the attributetree yields only sufficient conditions for one alternativedominating another alternative. It has the advantage,though, of providing a display of maximum and minimum

SAGE AND WHITE: KNOWLEDGE-BASED INTERACTIVE SYSTEM 45

Fig. 5. Domination digraph.

and we use the expressions on the right side of this

(7)N

~(a) = L Pj~i(a).1=1

where ~i(a) is the utility of thejth attribute of outcomestate i associated with alternative a and Pj is the trade-offweight associated with the jth attribute. Generally, thedecision situation should be structured such that the weightsare alternative and outcome state independent. If this isnot the case, there is very likely a modeling deficiency inthe framing of the decision situation structural model.

Combination of (6) and (7) results in

where M outcome states can result from alternative a. StateXi occurs with probability Pie a), and the utility of this stateis U, (a). This utility function will generally be a multiat­tribute utility function. When additive independence con­ditions are satisfied we have

B. Decisions Under Risk

For the decision under risk situation, we calculate theexpected utility of alternative a from

M

EU(a) = L Pi(a)~(a) (6)i=l

M N

EU(a) = L L Pi(a)~i(a)pj = pT(a)U(a)p (8)i=l.1=l

wheref'It c) and p are vectors of dimension M and N, andU(a) is an M X N vonNeuman Morgenstern cardinal util­ity function expressed as a matrix. Alternative a is guaran­teed to be preferred to alternative b if

min [pT(a)U(a) - pT(b)U(b)]p > 0 (9)A

where A represents the set of all possible values that theparameters p (a), p (b), U( a), U( b), and p can assume.

The simplest case occurs when probabilities and utilitiesare known precisely and only weights are imprecise. Weobtain A(A - 1) linear programs to solve for all possiblealternative preferences is the weight set inclusion is de­scribed by linear inequalities. We obtain necessary andsufficient conditions for preference inequalities. In a simi­lar way, if only the probabilities or only the across attri­butes are imprecise, we may solve a set of A(A - 1) linearprograms to obtain necessary and sufficient conditions for

inequality to determine the minimum and maximum scoreson each alternative for each aggregated attribute.

Using necessary and sufficient conditions for preferencedetermination is highly desirable. The way around thepotential dilemma noted earlier is to use the best ideal andworst ideal alternative concept as prompts to the de­cisionmaker. These always propagate through the tree asthe best and worst alternatives and will always be anchoredat attribute scores of one and zero. We use these as anchorsfor the weight elicitation effort. All computations are madeusing (3) and (4) such that we always obtain necessary andsufficient conditions for preference condition determina­tion.

Attribute 3 (or w~)

max score min score

TABLE IV

Attribute wi

0.8 0.6 0 00.4 0.2 0.8 0.60.44 0.24 1 10.268 0.14 0.7 0.51 1 1 10 0 0 0

max score min scoreAl ternative

dideal bestideal worst

ab

> min vT(ai)w- max vT(aj)wv(ai)' w u(a j ) . lV

scores across alternatives after each aggregation up thetree. This is not obtained through use of the necessary andsufficient conditions of (3) and (4).

One way to avoid the problem with the best and worstalternative scores on each attribute not being uniquelyanchored on one and zero after aggregating up the attri­bute tree may be to define an ideal best and an ideal worstalternative. The ideal best alternative will have a score ofone on each lowest level attribute, and the ideal worstattribute will have a score of zero on each lowest levelattribute. The DM should still specify the weight bounds0.2 ~ w~ ~ 0.4 and 0.6 ~ w~ ~ 0.8 obtained earlier. Nowthe aggregation up the attribute tree preserves the anchorover zero to one on alternative scores for the ideal alterna­tives, and we have Table IV. Elicitation of swing weightsmight now be more comfortably accomplished than in thecase where no pair of alternatives is uniquely anchored atzero and one on one or more aggregated attributes.

The question concerning whether the single-level attri­bute tree or the hierarchical tree is more appropriate in agiven situation is difficult to answer. Previous studies onthis point have not produced definitive guidelines. Ourexperience indicates that if the decisionmaker is comfort­able with the single-level tree and is willing to expressinformation concerning all attribute weights, then thisstructure is certainly more convenient to use and verylikely is more appropriate as well.

As we have indicated in Fig. 2, we can easily convertfrom one representation to the other. The only essentialdifference between the two approaches is that it is "natu­ral" when aggregating up the attribute tree to indicatemaximum and minimum scores on each alternative. Use ofthese to determine preferences results in only sufficientconditions for preference determination as we have

46 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. SMc-14, No.1, JANUARY/FEBRUARY 1984

where

alternative pair preferences if the precision is expressed asa set of linear inequalities.

If probabilities are known precisely, then we may rewrite(9) as

U(a) = minU(a),V(b) = maxU(b). (13)

This will enable us to obtain more desirable solutioncharacteristics. In case b), where probabilities are the onlyother imprecise parameters, we can solve a simple set oflinear programming problems to obtain necessary and suf­ficient conditions for alternative preferences.

In cases a) and c), where both probabilities and weightsare imprecise, it seems that no realistic way exists in whichto obtain solution for the preference inequalities by solvingsets of linear programming problems. This is a consider­able complication because of the solution complexity asso­ciated with nonlinear (quadratic) programming problemsand the fact that we usually will not obtain both necessaryand sufficient conditions for preference inequalities to hold.We can place a lower bound on the preference inequalitiesfor (9) in these cases. If this lower bound is greater thanzero, then we have sufficient but not necessary conditionsfor a given alternative preference. Even though we maydetermine the existence of alternative preferences through

where

yT(a, b) = min [pT(a)U(a) - pT(b)U(b)] (11)U(a), U(b)

and where the utilities of alternative a and b on the i thattribute are functionally independent of those on the jthattribute for i =1= j. Thus each column in (~1) can be opti­mized independently of one another. There are A(A - I)Nlinear programs to solve to specify (11) and A(A - 1)linear programs to determine the preference inequalities of(10) if parameter imprecision is expressed by linear in­equalities. We then can obtain necessary and sufficientconditions for preferences.

In all other cases,

a) utilities specified precisely, probabilities and weightsimprecisely;

b) weights specified precisely, probabilities and utilitiesimprecisely;

c) weights, probabilities, and utilities specified impre-cisely,

obtaining necessary and sufficient conditions for optimal­ity of linear programming solutions will generally not bepossible unless the imprecision can be expressed by meansof simple numerical inequalities, i.e., 0.2 ~ Pi(a) ~ 0.35.

In case b), we can obtain some desirable simplifications.Probabilities and weights are constrained by the sum toone property but utilities are not. When we have simplenumerical inequalities on utilities, we can rewrite (9) as

min [pT(a)U(a)-pT(b)U(b)]p~O (12)P(a), P(b), W

solution of linear programs, the fact that we establish onlysufficient conditions gives cause for concern as other suffi­cient conditions may well exist which yield considerablystronger preference relations.

From the foregoing discussion we see that it is a rela­tively straightforward matter to incorporate imprecision, inthe form of linear inequalities, into any combination ofutility scores for lowest level attributes, probabilities ofevent outcomes, and attribute weights as long as probabili­ties and weights are not simultaneously imprecise. If thisoccurs, we must solve quadratic programming problemsand are no longer able to get necessary and sufficientconditions for preferences.

ApPENDIX

The simplest forms of ARIADNE were designed for usein situations in which alternatives scores on lowest levelattributes and probabilities, if appropriate, are preciselyknown. The intent is to allow the decisionmaker to specifyprecise attribute weights in a bottom-up fashion so as to beable to aggregate up the attribute tree. Generally, this willresult in greater strength to the partial preference orderingamong alternatives. Ultimately, a scalar additive multiat­tribute utility (MAUT) function results. Although the pro­cess of obtaining this scalar MAUT function will generallybe quite different from that used in conventional MAUT,the substantive results should be the same. A rather com­plete discussion of this simplest decision-under-certaintyversion of ARIADNE is contained in [17].

Our initial efforts involving single-stage decision aidingunder uncertainty were based on precise specification ofnot necessarily all alternative scores on lowest level attri­butes and the use of stochastic dominance concepts. Aggre­gation up the attribute tree by means of elicitation ofpartial preference information concerning weights was usedto increase the strength of preference specificity. Twoforms of stochastic dominance have been considered, asdescribed in [18], [19].

The stochastic dominance concepts, especially second­order stochastic dominance, are computationally very timeconsuming for more than just a few attributes. We havediscovered and investigated a strong second-order stochas­tic dominance bound that greatly reduces computationalcomplexity. However, our research has shown that specifi­cation of bounds on parameters, such that linear orquadratic programming techniques may be used to identifya dominance structure, appears behaviorally much morerealistic as well as computationally much simpler thanstochastic dominance based approaches. Here, expectedvalue stochastic dominance is used in the uncertain case. Arather general description of the analytical constructs sup­porting this bounded inequality version of ARIADNE iscontained in [20].

Development of necessary and sufficient conditions foralternative preferences with parameter information con­straints expressed as linear inequalities is contained in [21].Further extensions of these analytical constructs, including

(10)minyT(a, b)p > 0w

SAGE AND WHITE: KNOWLEDGE-BASED INTERACTIVE SYSTEM

an inverse decision aiding approach to enable learning ofjudgmental weights in terms ofskill-based wholistic prefer­ences, are contained in [22]. The use of structural parame­ter imprecision concepts in the bottom-up and top-downdevelopment of attribute trees is described in [23], [24].

An overview of the concept is presented in [25]. Organi­zational, behavioral, and methodological concerns whichhave influenced system design are contained in [26]. Initialanalytical and algorithmic developments which serve as thebasis for extensions of ARIADNE to the sequential de­cisionmaking case are contained in [27]-[30]. Finally, anevaluation of the decision support system is discussed in[31].

REFERENCES

[1] J. Rasmussen, "Skills, rules, and knowledge: Signals, signs, andsymbols, and other distinctions in human performance models,"IEEE Trans. Syst., Man, Cybern., vol. SMC-13, May/June 1983.

[2] A. P. Sage, "Behavioral and organizational considerations in thedesign of information systems and processes for planning anddecision support," IEEE Trans. Syst., Man, Cybern., vol. SMC-11,pp. 640-678, Sept. 1981.

[3] A. P. Sage and E. B. White, "Decision and information structures inregret models of choice 'under uncertainty," IEEE Trans. Syst.,Man, Cybern., vol. SMC-13, pp. 136-145, Mar./Apr. 1983.

[4] H. Montgomery, "Decision rules and the search for a dominancestructure: towards a process model of decision making," GoteborgPsych01. Reps., vol. 11, 1981.

[5] A. P. Sage, "Methodological considerations in the design of largescale systems engineering processes," in Large Scale Systems, Y.Haimes, Ed. Amsterdam, The Netherlands: North Holland, 1982,pp. 99-14l.

[6] J. Pearl, A. Leal, and J. Saleh, "GODDESS: A goal directeddecision structuring system," IEEE Trans. Pattern Anal. MachineIntell., vol. PAMI-4, pp. 250-262, May 1982.

[7] D. W. Rajala and A. P. Sage, "On decision situation structuralmodeling," Policy Anal. Inform., Sci., vol. 4, pp. 53-81, July 1980.

[8] A. P. Sage, "On sensitivity analysis in systems for planning anddecision support," J. Franklin Inst., vol. 312, pp. 265-291,Sept./Oct. 1981.

[9] K. R. Hammond, G. H. McClelland, and 1. Mumpower, HumanJudgment and Decision Making. New York: Praeger, 1980.

[10] C. Argyris, Reasoning, Learning, and Action. San Francisco, CA:Jossey-Bass, 1982.

[11] A. P. Sage, Methodology for Large Scale Systems. New York:McGraw-Hill, 1977.

[12] I. L. Janis and L. Mann, Decision Making. New York: Free Press,1977.

[13] D. Kahneman, P. Slovic, and A. Tversky, Eds., Judgment UnderUncertainty: Heuristics and Biases. New York: Oxford Univ. Press,1982.

[14]

[15]

[16]

[17]

[18]

. [19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

47

B. Fischhoff, P. Slovic, and S. Lichtenstein, "Knowing what youwant: Measuring labile values," in Cognitive Processes in Choice andDecision Behavior, T. S. Wallsten, Ed. Erlbaum, 1980.P. Humphries, "Use of problem structuring techniques for optiongeneration: A computer choice case study," in Analysis and AidingDecision Processes, P. Humphries, O. Svenson, and A. Vari, Eds.Amsterdam, The Netherlands: North Holland, 1983.L. Zadeh, "A computational approach to fuzzy quantifiers in natu­ral language," in Computational Linguistics, N. 1. Cercone, Ed.Oxford: Pergammon, 1983.C. ·C. White and A. P. Sage, "A multiple objective optimizationbased approach to choicemaking," IEEE Trans. Syst., Man, Cybern.,vol. SMC-10, pp. 315-326, June 1980 (reprinted in R. Saeks and1. D. Palmer, eds., The World of Large Scale Systems. New York:IEEE Press and Wiley, 1981, pp. 335-346).__ , "Second order stochastic dominance for multiple criteriaevaluation and decision support," in Proc. 1981 Int. Conf. Cybernet­ics and Society, 1981, pp. 572-567.__, "Multiple objective evaluation and choicemaking under riskwith partial preference information," Int. J. Syst. Sci., vol. 14, pp.467-485,1983.C. C. White, A. P. Sage, and W. T. Scherer, "Decision support withpartially identified parameters," Large Scale Syst., vol. 3, pp.177-190, Aug. 1982.C. C. White, S. Dozono, and W. T. Scherer, "An interactiveprocedure for aiding multiattribute alternative selection," Omega,vol. 11, pp. 212-214, 1983.C. C. White, A. P. Sage, and S. Dozono, "Implications of imprecisemeasure of preference on alternative ranking specificity," in Proc.6th IFAC Symp. Identification and System Parameter Estimation,1982, pp. 592-597.C. C. White and A. P. Sage, "Imprecise importance weight assess­ment for multilevel objectives hierarchies," in Proc. IEEE Int. Conf.Large Scale Systems, 1982, pp. 298-30l.A. P. Sage and C. C. White, "A top down approach to impreciseinteractive attribute weight, structure, and alternative score assess­ment," in Proc. IEEE Int. Conf. Syst., Man, Cybern., 1982, pp.663-667.__, "Behavioral issues in a knowledge based interactive systemfor decision support," in Proc. IEEE Conf. Decision and Control,1982, pp. 1049-1052.A. P. Sage, "A methodological framework for systemic design andevaluation of planning and decision support systems," Comput.Elec. Eng., vol. 8, pp. 87-102, June 1981.C. C. White and H. EI Deib, "Multistage decisionmaking withimprecise utilities," in Essays and Surveys on Multiple CriteriaDecision Making. New York: Springer-Verlag, 1983, pp. 400-405.C. C. White, "Sequential decisionmaking with uncertain futurepreferences," Operations Res., to be published.C. C. White and H. EI Deib, "Parameter imprecision in finite state,finite action dynamic programs," submitted for publication.C. C. White, A. P. Sage, and S. Dozono, "A model of multiattributedecisionmaking and trade-off weight determination under uncer­tainty," IEEE Trans. Syst., Man, Cybern., to be published.C. C. White, A. P. Sage, S. Dozono, and W. T. Scherer, "Anevaluation of ARIADNE," in Proc. 6th MIT/ONR Workshop onCommand and Control, July 1983.