22
This article was downloaded by: [University of Connecticut] On: 08 October 2014, At: 06:40 Publisher: Taylor & Francis Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK Applied Artificial Intelligence: An International Journal Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/uaai20 Knowledge acquisition for knowledge based decision systems Jae-Kyeong Kim Published online: 26 Nov 2010. To cite this article: Jae-Kyeong Kim (1997) Knowledge acquisition for knowledge based decision systems, Applied Artificial Intelligence: An International Journal, 11:2, 131-150, DOI: 10.1080/088395197118280 To link to this article: http://dx.doi.org/10.1080/088395197118280 PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access

Knowledge acquisition for knowledge based decision systems

Embed Size (px)

Citation preview

Page 1: Knowledge acquisition for knowledge based decision systems

This article was downloaded by: [University of Connecticut]On: 08 October 2014, At: 06:40Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number:1072954 Registered office: Mortimer House, 37-41 Mortimer Street,London W1T 3JH, UK

Applied ArtificialIntelligence: AnInternational JournalPublication details, including instructions forauthors and subscription information:http://www.tandfonline.com/loi/uaai20

Knowledge acquisition forknowledge based decisionsystemsJae-Kyeong KimPublished online: 26 Nov 2010.

To cite this article: Jae-Kyeong Kim (1997) Knowledge acquisition forknowledge based decision systems, Applied Artificial Intelligence: AnInternational Journal, 11:2, 131-150, DOI: 10.1080/088395197118280

To link to this article: http://dx.doi.org/10.1080/088395197118280

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of allthe information (the “Content”) contained in the publications on ourplatform. However, Taylor & Francis, our agents, and our licensorsmake no representations or warranties whatsoever as to the accuracy,completeness, or suitability for any purpose of the Content. Anyopinions and views expressed in this publication are the opinions andviews of the authors, and are not the views of or endorsed by Taylor& Francis. The accuracy of the Content should not be relied upon andshould be independently verified with primary sources of information.Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilitieswhatsoever or howsoever caused arising directly or indirectly inconnection with, in relation to or arising out of the use of the Content.

This article may be used for research, teaching, and private studypurposes. Any substantial or systematic reproduction, redistribution,reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of access

Page 2: Knowledge acquisition for knowledge based decision systems

and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

6:40

08

Oct

ober

201

4

Page 3: Knowledge acquisition for knowledge based decision systems

KNOWLEDGE ACQUISITION FOR KNOWLEDGE-BASEDDECISION SYSTEMS

JAE-KYEONG KIMDepartment of Management Information Systems,Kyonggi University, Yiui-dong, Paldal-ku, Suwon,Kyonggi, Korea

Knowledge-based decision systems support the decision analysis process with decision

analytic knowledge as well as domain-specific knowledge. The knowledge in the knowledge-

based decision system is usually represented by influence diagrams. In the development of

knowledge-based decision systems, knowledge acquisition is one of the most difficult prob-

lems. This article suggests a knowledge acquisition process using verbal protocol analysis and

multi-attribute decision-making methodology. An environmental decision-making problem is

used as an illustrative example to explain the developed knowledge acquisition process. The

suggested process is so flexible that it can be expanded or applied to other problems with

ease. Some guidelines for the knowledge-based decision system are also suggested.

Everyone faces the challenge of making good decisions. Decision making is

complicated by competing alternatives, conflicting objectives, and uncertain conse-

quences. Uncertainty, in particular, makes decision making difficult, whether it

concerns the profitability of a new product or the health risk of toxic chemical

testing. Decision theory is an axiom-based framework that explicitly considers

uncertainty and trade-off. Decision analysis (DA) is an engineering discipline that

tells how to apply decision theoretic principles to real problems in a tractable manner

(Howard, 1984, 1988; Howard & Matheson, 1984). Since the early 1980s when

several knowledge-based systems proved to be successful, there have been some

approaches to apply knowledge-based systems to decision analysis (Chung et al.,

1992; Holtzman, 1985; Howard, 1988; Kim, 1991; Kim et al., 1990, 1992; Kim &

Kim, 1991; Reed, 1989; Russel et al., 1988). A knowledge-based system that

implements the decision analysis is referred to as the decision analysis expert system

(Howard, 1988), the intelligent decision system (Holtzman, 1985, 1989; Kim, 1995;

McGovern et al., 1991) or the knowledge-based decision system (Chung et al., 1992;

Kim, 1991; Kim et al., 1992; Park et al., 1993). The common properties of these

systems are that they provide their users with a substantial amount of domain-

specific knowledge and normative power of decision analysis. In this article, the

Applied Artificial Intelligence, 11:131± 149, 1997Copyright � 1997 Taylor & Francis0883-9514/97 $12.00 + .00 131

This work was done when the author was in the IDS department, University of Minnesota as a visiting faculty

member. The author wishes to thank Professor Carl R. Adams for his careful review and comments on content and

style, and Melissa L. Lange for her help on the interview and analysis of the verbal protocol.

Address correspondence to Jae-Kyeong Kim, Department of Management Information Systems, Kyonggi

University, San 94-6, Yiui-dong, Paldal-ku, Suwon, Kyonggi, Korea.

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

6:40

08

Oct

ober

201

4

Page 4: Knowledge acquisition for knowledge based decision systems

knowledge-based decision system (KBDS) is used as a representative tool for

decision making. A KBDS constitutes a means by which decision makers can exploit

the normative power of decision analysis in a relatively simple and inexpensive way.

In the development of KBDS, there is no general agreement on the use and

appropriateness of the representation scheme and various measures of uncertainty.

It is, however, widely recognized that influence diagrams are used to represent

domain knowledge, and probability inference using Bayesian theory is used to

handle uncertainty (Chung et al., 1992; Holtzman, 1985, 1989; Howard, 1988; Kim,

1991, 1995; Kim et al., 1990, 1992; Park et al., 1993). The concept underlying

influence diagrams is the representation of variables critical to the problem under

consideration, and their interrelationships, in a graph-theoretic manner (Howard &

Matheson, 1984; Olmsted, 1983). Much of the power of decision analysis lies in its

ability to effectively integrate the many factors that commonly affect a decision, so

influence diagrams are used as a knowledge representation scheme of KBDS in this

research.

Domain-specific knowledge of KBDS is represented as influence diagrams,

which consist of decision variables, uncertainty variables, criteria values, and their

relationships (Holtzman, 1985, 1989; Kim, 1991; Kim et al., 1990, 1992). However,

it is not an easy problem for the decision makers and decision analysts to elicit (and

define) many variables and integrate them effectively to structure a decision prob-

lem. These difficulties caused by the complex and evolutionary nature of the

structuring process are considered to be a bottleneck to the widespread use of

decision analysis as well as to the development of KBDS (Holtzman, 1985, 1989;

Howard, 1988; Kim, 1991). Another problem is that there has been little research

on how to modify an existing knowledge base or model base of KBDS to solve

similar problems (Kim et al., 1990; Owen, 1984). The amount of effort, money, and

time spent formulating a decision problem into an influence diagram is very

burdensome. It is also difficult to apply the knowledge or information obtained while

formulating one decision problem into other similar problems. This is another

difficulty to the development of KBDS (Holtzman, 1989; Howard, 1988).

It is not easy to find a research resource on the implementation of KBDS,

especially on the knowledge acquisition of that system. Research on the influence

diagram structuring process surveyed in this research is related to the knowledge

acquisition process of KBDS. RACHEL is the first KBDS to diagnose infertility

(Holtzman, 1985, 1989). It defines a class of decisions to build a domain-specific

system for that class. However, RACHEL is a character-based system, having no

graphic user interface. KIDS is for strategic decision problems by HyperCard

(Chung et al., 1992; Kim, 1991; Kim et al., 1992). It has a graphic user interface and

knowledge base to structure an influence diagram. The main limitation of this system

is its limited knowledge base capacity. The interview process (Holtzman, 1989;

Howard, 1988) and the goal-directed approach (Holtzman, 1989; Kim et al., 1990;

Newell & Simon, 1977) are frequently used to structure an influence diagram in the

132 J.-K. Kim

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

6:40

08

Oct

ober

201

4

Page 5: Knowledge acquisition for knowledge based decision systems

decision-making process. The interview process is a kind of conversation process

interchanging domain-specific knowledge and decision-analytic knowledge be-

tween decision makers (or domain experts) and decision analysts. One of the main

purposes of a knowledge-based system (including KBDS) is to mimic or replace an

expert (Kim, 1991). Thus the interview process alone is not suitable for the KBDS

because it needs the decision analyst(s) at all times. INFORM by Moore and Agogino

(1987; Russel et al., 1988) focuses on editing the knowledge base using a domain

independent tool. The induction of influence diagrams from a set of examples is

conducted in the domain of strategic management constructing a knowledge base.

McGovern et al. (1991) suggest the elicitation of decision structures through text

analysis. They emphasize that this approach is primarily exploratory with less well

structured problems. They argue that when combined with natural language process-

ing software, it will be possible to acquire knowledge of KBDS. Most real decision

problems are very diverse and slightly different even within similar problem

domains. Thus the knowledge from text alone is not suitable for the KBDS.

In this research, we develop a knowledge acquisition process based on verbal

protocol analysis (VPA) and multiple-attribute decision making (MADM) for the

development of KBDS. The VPA has been extensively applied to understanding

human problem solving to identify the problems and current decision-making

processes (Ericsson & Simon, 1984; Newell & Simon, 1977). VPA is used in this

research because it is a powerful tool for extracting knowledge from human experts

when they solve real managerial decision problems with their heuristic knowledge

combined with decision-analytic knowledge and domain-specific knowledge. VPA

makes it especially easy to transfer the extracted knowledge to a knowledge base

using a well-defined coding scheme. Merkhofer (1990; Merkhofer & Keeney, 1987)

suggests the use of an influence diagram to solve MADM problems. His idea of

combining an influence diagram and MADM is used in this research to model a

complex decision problem and to apply the developed model to other similar

problems. The main ideas proposed in this article can be summarized as follows.

1. VPA helps to determine the scope or boundary of the decision problems. The

analysis makes it possible to capture abstract knowledge of the influence diagram

and related information with ease, even if decision makers have their own heuristic

knowledge, rather than decision-analytic knowledge.

2. A hierarchical-type MADM makes it easy to classify the knowledge of

components or objectives of decision problems. Influence diagrams combined with

MADM methodology make it possible to enlarge or modify an existing model to be

adjusted to similar problems. MADM can also handle well the trade-off between

conflicting objectives.

3. This study provides the expansion procedure from a high-level influence

diagram (i.e., general, rough, or abstract-level influence diagram) to a low-level

influence diagram (i.e., more detailed or concrete influence diagram). This proce-

Knowledge Acquisition for Knowledge-Based Decision Systems 133

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

6:40

08

Oct

ober

201

4

Page 6: Knowledge acquisition for knowledge based decision systems

dure is very helpful for modeling of complex decision problems. It also makes it

easy for decision makers (including domain experts) to make their abstract

knowledge more concrete.

The process is explained by applying it to an environmental decision problem.

Some guidelines are suggested in regard to how to transfer or modify the knowledge

base for another similar problem. In the next section, the basic concept of influence

diagrams with inference problems is provided, and the knowledge acquisition

process, and VPA are described. Then, building an influence diagram, expansion

procedure, and transferring knowledge between decision problems are explained for

an environmental problem. Difficulties of developing KBDS and guidelines to

overcome them are also discussed. Finally, conclusions and possible future research

areas are provided.

KNOWLEDGE REPRESENTATION

There have been many approaches to implement the decision analysis process

with the help of computers. A KBDS applies the descriptive ideas of knowledge-

based system theory at a higher level, so it can manage the process of formulating,

evaluating, and appraising a decision problem. The knowledge of this system would

be knowledge for the creation of decision models, knowledge for their evaluation,

and knowledge for explaining their implications (Holtzman, 1985, 1989). Although,

the human decision analyst is too general-purpose to be captured everywhere, the

KBDS can be built only within a particular domain of application. An influence

diagram is used as a knowledge representation tool and inference tool of KBDS in

this research.

Influence Diagrams

An influence diagram is a major knowledge representation tool of KBDS. It was

developed as a computer-aided modeling tool by Miller et al. (1976) and Howard

and Matheson (1984). Olmsted (1983) proposed a computational architecture for

automating influence diagrams along with rules specifying the topological transfor-

mations needed for his solution procedure. Shachter (1986) developed a goal-

directed algorithm based on these rules. In contrast to other models, influence

diagrams focus decision makers’ attention on only those problem features that are

most relevant to the focus at hand. Excluding irrelevant information from an

influence diagram can save decision makers time and effort, since there are fewer

variables to be interpreted. If decision makers want to pay attention to a certain part

more specifically, they can expand part of an influence diagram to a lower level.

The flexibility of influence diagrams makes it easy for decision makers to communi-

cate with each other and to store influence diagrams for future use.

134 J.-K. Kim

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

6:40

08

Oct

ober

201

4

Page 7: Knowledge acquisition for knowledge based decision systems

An influence diagram is defined as a network consisting of a directed graph with

no directed cycles. The diagram consists of nodes and arcs (influences, or arrows)

that represent a decision basis (Howard & Matheson, 1984; Howard, 1988). Each

node in the graph represents a variable in the model. This variable may be a constant,

an uncertain quantity, a decision to be made, or a criterion value. The nodes can be

viewed from three levels: relational, numerical, and functional (Holtzman, 1985;

Howard & Matheson, 1984; Kim et al., 1992). Figure 1 shows an influence diagram

for the decision problem of a raw material buyer (Chung et al., 1992; Kim, 1991;

Kim et al., 1992). The nodes and arcs represent the relational level, i.e., showing the

major components, their types, and their interdependence.

In Figure 1, ª amount of oil purchaseº is a decision node. Decision nodes

represent decision options that are available to a decision maker (DM). The arcs to

a decision node represent information available at the time the decision is made.

Each decision node denotes variables under the control of the DM. It has branches

to represent the possible options. Each chance node has an underlying probability

distribution to quantify the uncertainty for the variable represented by that node.

Arcs into chance nodes represent information affecting the probability distribution

for that node. Chance nodes are generally represented as a cycle or an oval in the

influence diagram. ª Total oil priceº is a value node, and it summarizes the preferen-

ces of the DM for the decision outcomes. A mathematical function (value function

or utility function) associated with a value node can be used for deriving a numeric

value representing the trade-off among attributes of the problem. Direct predecessor

Figure 1. Influence diagram for the oil purchase problem.

Knowledge Acquisition for Knowledge-Based Decision Systems 135

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

6:40

08

Oct

ober

201

4

Page 8: Knowledge acquisition for knowledge based decision systems

nodes of a value node indicate the attributes included in the evaluation of preferen-

ces. See Olmsted (1983) and Howard and Matheson (1984) for more details about

influence diagrams. The next subsection discusses the probabilistic inference using

influence diagrams containing only chance nodes.

Inference Problem Using Influence Diagram

A probabilistic influence diagram is a special type of influence diagram that

contains only chance nodes. Once a problem is modeled using the influence diagram,

problem analysis and probabilistic relationships can be manipulated at the graph

level of the diagram. In the KBDS, knowledge inference can be made if the decision

maker needs a more detailed analysis. In 1988, Shachter showed how any prob-

abilistic inference problem can be solved through basic operations on an influence

diagram (Schachter, 1988). Also, Rege and Agogino (1988) formally defined the

consistent transformation of influence diagrams, probabilistic inference problems,

and a polynomial-time algorithm to solve the same transformed problem at a

symbolic and topological level. In this section, a brief explanation of probabilistic

inference is given.

Consider n state variables x1, x2, . . . , xn involved in the decision problem being

modeled. Each variable represents a factor or component of the problem, which has

a finite set of possible outcomes and a conditional probability distribution over those

outcomes. The information about the problem can be represented in terms of a joint

probability distribution of all variables denoted by P(x1, x2, . . . , xn). Such a joint

distribution can be expanded into any of n! expansions. The following equation

represents one of the possible expansions: P(x1, x2, . . . , xn) = P(x1)P(x1|x2), . . . ,

P(xn|x1, x2, . . . , xn ± 1).

Shachter’ s (1988) probabilistic influence diagram is an influence diagram contain-

ing only nodes that represent constant or uncertain quantities. A probabilistic influence

diagram represents one expansion of the joint distribution of the state variables by means

of a directed graph. Each variable in a probabilistic influence diagram has a data frame

within its associated node, in which there is a finite set of outcomes, and a conditional

probability distribution over those outcomes. The conditional relationship of each node

is represented by arcs in the diagram. If there is no undirected path between two nodes,

then they must be independent. If arc (i, j) is part of the diagram, then the assessed

distribution for the variable j is conditional on the value of i. When a chance node has

no arcs to it, then the assessed distribution is a marginal distribution. There are two types

of variables in a probabilistic influence diagram. A deterministic variable has a degen-

erate conditional distribution. If it does not, it is called a probabilistic variable. We might

be uncertain about the value of a probabilistic variable even after observing the values

of its conditioning variables. In the case of a deterministic variable, we are certain of its

value, given the value of its conditioning variables, although we might be uncertain

about its value if we could not observe their values.

136 J.-K. Kim

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

6:40

08

Oct

ober

201

4

Page 9: Knowledge acquisition for knowledge based decision systems

The general probabilistic inference problem is to determine P[fo(xj)|xk], where

J and K are arbitrary subsets of set N and fo is an arbitrary real-valued function

defined by the outcomes of xj given an influence diagram G = (N, A), where N is a

node set and A is an arc set. It can also be represented as P(x i|x j), where i is an arbitrary

node in the node set N. A probabilistic database for the variables xN is assumed to

have been assembled in the form of a partially specified influence diagram (Shachter,

1986). For example, suppose the OPEC policy is changed and the raw material buyer

of a textile cooperation wants to know the likelihood that the oil price in the spot

market will increase given that condition. If such a problem is represented as in

Figure 1 except for the decision and value node, he/she should solve the probability

inference problem to obtain P(Spot Price | OPEC Policy). Shachter (1986) and Rege

and Agogino (1988) developed a very similar reduction algorithm for probabilistic

inference problems. They use node reduction and arc reversal procedures, originated

from Howard and Matheson (1984) and Olmsted (1983). See Matzkevich and

Abramson (1995) for a survey of related studies.

KNOWLEDGE AC QUISITION

Knowledge Acquisition Process

The notion of mental representation is the principle that unifies research refer-

ences to problem setting, problem conceptualization, locating the problem, require-

ment determination, and framing and structuring decisions (Smith, 1988, 1993).

Each of these activities involves development of a mental representation of a

situation that deserves attention, thought, and action. Cognitive scientists claim that

having a mental representation of a problem is a necessary condition for trying to

solve it (Newell & Simon, 1977; Smith, 1988). Such representations can be exter-

nalized using various symbolic media. An influence diagram is one method for

external representation of uncertain, complex decision problems. Decision analysis

is regarded as a systematic procedure for transforming opaque decision problems

into transparent decision problems by a sequence of transparent steps (Howard,

1988). Opaque means ª hard to understand, solve, or explain; not simple, clear, or

lucid,º and transparent means ª readily understood, clear, obvious .º In other words,

decision analysis offers to a DM the possibility of replacing confusion by clear

insight. The transformation is generally composed of formulating, evaluating, and

appraising the decision problems.

The first step of a formulation fits a formal model to the DM’ s opaque real

situation. This formal representation of the decision problem is usually represented

as an influence diagram. The formulation process (or other process) is usually

conducted through conversation between the DM (and domain experts) and a

decision analyst (or knowledge engineer). During the formulation process, the DMs

or domain experts teach the details of the decision at hand (e.g., available resources,

Knowledge Acquisition for Knowledge-Based Decision Systems 137

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

6:40

08

Oct

ober

201

4

Page 10: Knowledge acquisition for knowledge based decision systems

attitude toward risks, or conflicting objectives) to the decision analyst, who learns

by building an appropriate influence diagram (Holtzman, 1985, 1988). This conver-

sation seems to be inevitable when DMs confront difficult decision problems, and

they have an uncanny ability to list the factors relevant to their problems or their

relationships. However, people who face similar decision problems again and again

have their own heuristic decision-making process by learning from experience, right

or wrong.

Based on previous research, the key points of knowledge acquisition of KBDS

may be summarized by the following two facts. In contrast to decision-analytic

knowledge, which is almost permanent, domain-specific knowledge is changed very

rapidly from situation to situation. First, domain-specific knowledge of a problem

should be obtained from human domain experts or DMs. Second, after the

knowledge is stored in the knowledge base of KBDS, similar decision problems

should be solved from the existing knowledge base after a slight extension or

modification. This research thus suggests a knowledge acquisition process based on

VPA and MADM. The main purpose of VPA is to obtain a clear understanding of

the current decision-making process. Designing a better process starts from a clear

understanding of the current process. The VPA technique has been extensively

applied to understanding human problem solving in order to identify the problems

and current decision-making process (Ericsson & Simon, 1984; Newell & Simon,

1977). VPA is not only a powerful tool extracting knowledge from human experts

but it also makes it easy to transfer the extracted knowledge to a knowledge base

using a well-defined coding scheme. MADM can treat well the relationship and

trade-off of objectives, so an influence diagram combined with MADM is very

useful. Furthermore, in a hierarchical structure, two similar decision problems are

identical at a higher level even though they are slightly different at a lower level.

Thus a similar decision problem can be easily formulated from the high-level

influence diagram in the knowledge base of KBDS.

Verbal Protocol Analysis

To elicit knowledge from DMs or (domain) experts, it is essential to gain a clear

understanding of the current decision situation and decision-making process. Verbal

protocols are verbal reports obtained from experts who have been instructed to

ª think out loudº while solving problems. DMs and experts are requested to express

their thoughts step-by-step while solving the decision problems presented. VPA

identifies the states that are explored and the knowledge that is accessed and applied

to transform one state into another. The results of VPA are used to constrain the

normally unconstrained task of knowledge representation design. They are given a

decision problem, which may be new or similar to one seen before. They are

requested to express their thoughts step by step into a tape recorder while solving

the decision problems provided. If there are aspects of the current decision-making

138 J.-K. Kim

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

6:40

08

Oct

ober

201

4

Page 11: Knowledge acquisition for knowledge based decision systems

process with which a DM is not satisfied, it is important to identify these in order to

design a new process. It is also important that the DM mention instances of decisions

made, largely or in part, based on rules of thumb, intuition, emotions, or external or

political constraints rather than by using a predefined decision-making process.

After protocols from DMs are collected, they are transcribed and analyzed

according to a well-defined developer-specific coding scheme. An illustrative

example of using protocol analysis is shown in the next section. The following

guidelines are suggested to DMs for the protocol analysis of this research.

1. Look carefully at the attached decision problems. You may have encountered

this decision previously, yet attempt to look at it as if you were seeing it for the first

time.

2. Do not think through the problem fully and then begin recording your

problem-solving process. It is the process of ª thinking through the problem fullyº

that we wish you to record. Look at it once, and then express your thinking process

precisely and step by step into the tape recorder. Please be as frank and thorough as

possible.

3. The thinking process that is to be recorded begins when you first look at the

problem, and ends once you have made your decision for a given problem.

4. The most important point is to express what you are thinking. W hat are you

taking into consideration? How do these factors relate to each other?

The VPA is frequently used to obtain information regarding a subject’ s thoughts

while focused on a problem-solving task (but not on the subjects’ interpretation of

these thoughts) (Ericsson & Simon, 1984; Newell & Simon, 1977). This method is

frequently used to gather information about the cognitive processes of a subject.

System developers (the author, in this research) analyze the verbal protocol obtained

from the recording machine. In such cases the data are ª encoded.º Encoding is the

process for translating raw protocols into a form suitable for analysis by the various

models being tested. In this research, the data were encoded mainly to seek answers

to the following questions: W hat is the decision problem at hand? What is being

decided? What are the objectives that need to be achieved through the decision?

What information is needed (and is it obtainable) as one proceeds? What and where

are the main uncertainties and risk factors? What is the approach used in weighing

the issues in relation to each other?

The respondents were asked to talk into a tape recorder while solving the

decision problem. The decision problem used in this case differs from tasks typically

used in VPA because the subjects cannot entirely solve the task at hand, since its

solution depends on interactions with other sources. The respondents could, how-

ever, arrive at their own conclusions without difficulty. The VPA provides useful

knowledge of the disjoined nature of thinking out loud. Something a DM says at one

time may contradict something said earlier or later. It is a very helpful technique to

Knowledge Acquisition for Knowledge-Based Decision Systems 139

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

6:40

08

Oct

ober

201

4

Page 12: Knowledge acquisition for knowledge based decision systems

elicit knowledge about the problem domain and the decision-making process. It also

helps to draw a high-level influence diagram easily and obtain some unconnected

detailed information.

ENVIRONMENTAL REVIEW DEC ISION MAKING: AN ILLUSTRATIVE EXAMPLE

Endangered Species Problem

The Minnesota Department of Natural Resources (DNR) considers a project

proposal in the environmental review process. Their considerations when they

review a project proposal are as follows: the legal status of the species, the viability

of that instance of the species, and effects of changes and species loss on the larger

ecological community picture. They usually also consider the social and economical

benefits and the consequences or legal repercussions for denying the permit. But

their main concerns are what effects the project will have on the species on site and

on the state of Minnesota. They encounter this decision about two or three times a

year. DNRs in other states consider similar problems. Some of the previous and

current decision situations are summarized in Table 1.

The decision situations described in this research are those that assist in deciding

between two possible, yet mutually exclusive courses of action, both resulting in a

win-lose situation. Periodically, the Minnesota DNR Natural Heritage and Nongame

Research Program encounters situations that are similar but vary in location, project

type, and number and type of species threatened (Table 1 depicts some of these

decision situations). In previously encountered cases, the decision-making process

occurred on a basis of heuristic knowledge. The DMs did not always follow an

explicit set of criteria but have their own criteria, which are slightly different from

person to person. This made it difficult to consider the multiple criteria effectively

and efficiently, and to defend the process. Some DMs also did not have an explicit

approach for assessing the relative importance of various factors or for evaluating

Table 1. Diverse decision problems of the Department of Natural Resources

Project Considerations

Landfill Expansion Existing landfill expanded onto 63 acre site

· Proposed site 17 acres of prairie, 20 acres of wetlands, 1 endangered and 1 threatened plant species

· Crucial considerations Quality and rarity of site, social pressures

Burnsville Flood Control Impound a lake in order to dam an area to prevent flooding in downstream areas

· Proposed site Very rare MN endangered plant; habitat found also very rare

· Crucial considerations Many people involved, large momentum of project, precedent the decision will set

Giants Ridge Build a large-scale development and golf course next to existing ski resort area

· Proposed site 3 endangered plant species, 2 threatened plant species, 1 plant of special concern

· Crucial considerations Forest fragmentation and water quality; no permits granted to date for a

development project

140 J.-K. Kim

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

6:40

08

Oct

ober

201

4

Page 13: Knowledge acquisition for knowledge based decision systems

trade-offs. The decision of whether to grant or deny can be solved using decision

analysis methodologies. The DNR DM has domain-specific knowledge and decision-

analytic knowledge, but the whole decision-making procedure is not well developed.

Knowledge acquisition for a landfill expansion project is performed below using

VPA and MADM.

Knowledge Acquisition Using VPA

DMs are requested to express their decision-making process and domain-specific

knowledge in as much detail as possible into a tape recorder. Protocols from four DMs

are collected for this study. They are analyzed from sentence to sentence, the sentences

are grouped by their similarities, and some assertions are elicited. These assertions are

helpful in understanding the problem and the process. The following are some assertions

and supporting protocol fragments. The number preceding each sentence represents

sentence sequences by the author’ s encoding scheme.

Assertion 1: First, DMs want to understand the problem, its decisions, and their

objectives.

DM 1:

01 I assume that the problem we are looking at is where to issue the permit of

the species that are listed . . . to be specific, what we are deciding is whether to issue

a permit for the taking of two state-threatened plant species that are known to occur

on this site.

02 The objectives that have to be taken into consideration are the protection of

endangered species in this state, preventing them from becoming extirpated but also

the integrity of the endangered species act so that it will be available for future use

as well as present use. . . . umm.

DM 3:

01 A landfill expansion . . . with 17 acres of prairie, 15 plant, 20 acres wetland.

02 It seems obvious that in the metro area, a 20-acre wetland to be proposed to

be filled would require a 404 Water Quality Permit from the Corps of Engineers and

quite likely a permit from the DNR under the Wetlands Conservation Act. . . .

04 The 17 acres of prairies have no legal protection; there is no jurisdiction here

by the DNR.

Assertion 2: DMs extract major uncertainty factors first, and other related fac-

tors step by step.

DM 4:

08 Um. The main uncertainties and risk factors. . . . I am using the bullets from

the sheet to talk about this process. . . . One big uncertainty is what the long-term

Knowledge Acquisition for Knowledge-Based Decision Systems 141

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

6:40

08

Oct

ober

201

4

Page 14: Knowledge acquisition for knowledge based decision systems

viability of the prairie and the species on this site would be if the project doesn’ t go

forward.

13 Another uncertainty . . . is what sort of mitigation to ask for . . . how to go

about determining what the appropriate mitigation is, and how to successfully

negotiate with the project proposer to get it, are uncertainties that have to be dealt

with. . . .

DM 2:

15 It is of extremely high quality and it is the habitat of two threatened plant

species both of which have very well established and viable populations at this site.

16. And this landfill would not simply take part of the prairie and few of the

plants but would destroy all of the plants and all the habitat.

Assertion 3: DMs consider several objectives and related factors of each objec-

tive, and synthesize the relationships among objectives using trade-offs.

DM 1:

27 . . . my approach in weighing the different issues in relation to one another. . . .

28 . . . look at the significance of the plant species on the site and the prairie in

the context of the state as a whole.

DM 3:

32 So my first reaction is just to say no.

34 Ah . . . however, certainly there has to be considered, there is a need for more

landfill space . . . it would be economically and politically easier to simply expand

the current landfill into this area than it would be to sight an entirely new landfill

somewhere else. Opening landfills are very, very difficult.

Besides the above decision-making assertions, the elicited properties of this

problem are listed as follows:

· Main decision is made by several DMs after considering the trade-off between

viability at that instance of that species with social and economic benefits.

· The major uncertainty of the project that should be considered by DNR is the

probability of existence of species on site or in state.

· Another uncertainty is the projected viability of the population on site proposal

is permitted or not.

· Decisions of a similar type are to be made frequently, and these decisions strongly

depend on the specific situation of the site and project. Thus, if an influence

diagram is structured, it can be used repeatedly with a slight change of structure

and information.

· The species information, such as threatened, endangered, of special concern, total

number at site and in state, and density, can be stored in the database or knowledge

base of KBDS.

142 J.-K. Kim

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

6:40

08

Oct

ober

201

4

Page 15: Knowledge acquisition for knowledge based decision systems

Building an Influence Diagram from VPA and MADM

MADM is well suited for comparing options against multiple objectives.

Competing objectives, such as ª to maintain biodiversity of Minnesota and its social

benefitsº and ª to increase economic and other social benefits,º are common in

environmental problems. MADM establishes attributes for measuring the degree to

which identified objectives are achieved. On the basis of the protocol analysis, we

set up three objectives and related attributes of DNR decision problems.

1. To maintain biodiversity of the state

· Ensure the continued existence of species on site

· Ensure the continued existence of species in state

2. To increase the social benefit of state residents

· Increase social benefit from this site

· Increase economic benefit from this site

3. To abide by practical considerations

· Save time and cost of knowledge elicitation and decision making

· Avoid lawsuits

· Give consistent response to applicants.

On the basis of this hierarchical value structure and VPA, the following steps

are used to develop an initial high-level influence diagram in this research.

Step 1: Isolate objectives from verbal protocols

· Biodiversity in state (to maintain biodiversity of state)

· Benefit (to increase the social benefit of state resident)

· DNR (to abide by practical considerations)

Step 2: Isolate decision elements from verbal protocols

· Permit decision

· Viability in state

· Viability of the population on site

· Lawsuit

· Result of suit

Step 3: Establish relationships between elements from verbal protocols (the relation-

ships between elements are listed in pairs)

· Permit decision influences benefit

· Permit decision influences viability of the population on site

· Lawsuit influences result of suit

Step 4: Draw an influence diagram. Each node is examined, and its type either

decision, chance, or value node, is determined.

As a result, Figure 2 is constructed.

Knowledge Acquisition for Knowledge-Based Decision Systems 143

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

6:40

08

Oct

ober

201

4

Page 16: Knowledge acquisition for knowledge based decision systems

Expansion Procedure of an Influence Diagram

The major uncertainties of this problem (see Figure 2) are represented as the

following inference problem: P(viability of the species on site|proposal permit) and

P(viability of the species on site|proposal denial). Because this probability is not

easy to elicit directly from DMs, we expand this node into a lower level through an

interview process with DMs. Figure 3 represents part of such an expanded low-level

diagram.

Figure 4 represents the final version of the influence diagram of the landfill

project problem after the interview with decision makers. The main purpose of the

interview is to obtain the exact knowledge of a probability distribution and preferen-

ces from DMs. But to fulfill such a purpose , a more detailed influence diagram is

helpful. Building a final diagram currently occurs by obtaining all the knowledge.

A major benefit of influence diagrams comes when it is time to analyze the model

after building. Because an influence diagram is logically precise, there is no need to

transform the diagram into a form appropriate for analysis. The same graphical

representation that is so effective for modeling is also effective for analysis. Because

Figure 2. High-level influence diagram of the Department of Natural Resources.

144 J.-K. Kim

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

6:40

08

Oct

ober

201

4

Page 17: Knowledge acquisition for knowledge based decision systems

transformations of influence diagrams correspond to applications of probability

calculus, graphical techniques can be used to transform the diagrams to more

convenient forms without getting bogged down in mathematical details. See Olm-

sted (1983) and Shachter (1986) regarding transformations.

Transfer Knowledge Between Decision Problems

The Minnesota Natural Heritage Program must periodically make decisions

requiring transfer of knowledge from other problems. Analysis of three previous

decisions (in Table 1) indicates that the situations are similar, varying primarily in

the project type and location and in the number and type of species that would be

affected.

The Burnsville Flood Control Project and the Giants Ridge Project can be

modeled with ease from the decision model shown in Figure 4. The models are

relationally very similar because many variables are common between the problems.

However, the functional level or the numerical level will be somewhat or very

different. For example, we are to model the Burnsville Flood Control Project from

the Landfill Expansion Project. The influence diagram of the Burnsville project is

identical to that of the landfill project on the relational level. Thus Figure 4 can be

used for the Burnsville project. Furthermore, the options of the decision node are

same. The species database and knowledge base for the entire state will be used

without modification, but the species to be considered are different in the two

Figure 3. Expansion of one-chance node.

Knowledge Acquisition for Knowledge-Based Decision Systems 145

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

6:40

08

Oct

ober

201

4

Page 18: Knowledge acquisition for knowledge based decision systems

Figure 4. Influence diagram of landfill problem.

146

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

6:40

08

Oct

ober

201

4

Page 19: Knowledge acquisition for knowledge based decision systems

projects. Thus possible state and probability of chance node and value equations are

totally different from those of the landfill problem. DMs or domain experts may feel

confident in giving a subjective probability to a specific situation given the whole

structure when they face a similar problem. The DMs can solve the problems without

the help of decision analysts or knowledge engineers. The KBDS helps the DMs to

see the problem in a global sense and provides guidelines to elicit a situation-specific

knowledge (Kim, 1991; Park et al., 1993).

GUIDELINES TO KBDS

The following four guidelines are suggested for using KBDS for managerial

decision problems. The basis of these guidelines comes from existing studies.

First, it should be identified whether the problems are to be solved by decision

analysis, to determine if an influence diagram would be a suitable modeling tool.

Similar decision problems should occur frequently. For these kinds of problems, the

KBDS is a decidedly efficient system, saving effort, money, and time for decision

analysts and DMs (Kim, 1991).

Second, similar problems can be conveniently treated as one unit: a class of

decisions. Holtzman (1985, 1989) first suggested the concept of decision class

analysis. He defines a class of decisions as a set of decisions having some degree of

similarity that can be treated as a single unit. The decisions in a class share a common

domain and, perhaps, also some common situation features. Just as the end result of

a specific decision analysis is a decision, the end result of a decision class analysis

is a specific decision analysis. This is appropriate in small-frame situations. How-

ever, Holtzman does not explain how to construct a class of decisions and how to

decide whether to include a new problem in a certain class or not. The concept of

decision class analysis is unclear in some respects. The concept of a class of

decisions is very important but very difficult to define. The next problem is the

methodology of implementing the DCA. Holtzman used a rule-based approach,

but a rule-based approach is well suited to a narrow decision scope in the author’ s

experience (Chung et al., 1992; Kim et al., 1992). His idea of decision class

analysis is, however, thought to be the first stage in automating the decision

analysis process.

Third, the knowledge elicitation of a problem should be conducted by a

goal-oriented approach. In this article a mixture of protocol analysis and interview

process is suggested. A MADM is thought to be a good methodology for a complex

goal structure. A protocol analysis represents defining current problems, scope,

relationship of factors, needed information, preference, goal structure, and view-

point of the problem. Using a verbal analysis, it is possible to construct a high-level

influence diagram and other knowledge, as we have shown before. If necessary, a

node can be expanded to make it easy to elicit knowledge from DMs. This procedure

Knowledge Acquisition for Knowledge-Based Decision Systems 147

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

6:40

08

Oct

ober

201

4

Page 20: Knowledge acquisition for knowledge based decision systems

saves time and effort and avoids the need for trial and error if the problem is too

complex.

Fourth, decision problems of the same class are solved with ease using the initial

influence diagram because class problems have similar boundaries and share many

of the problem parts. After defining and solving one class of decision problems,

case-based reasoning (CBR) is thought to be a useful tool if KBDS are required to

seek automatic classification procedures (Beck et al., 1994). CBR is a technique that

compares new individual decision problems with an existing class of problems

stored in a knowledge base. This comparison is made on the basis of similarity

defined in terms of the problem structure. Individual decision problems may be

grouped into a new class based on similarities in their structures, regardless of any

existing class description, if they cannot fit into existing classes. Thus this kind of

classification algorithm may be a good alternative to the knowledge acquisition

process of KBDS.

CONC LUSION

This article presents a knowledge acquisition process using verbal protocol

analysis for the development of KBDS. The influence diagram is used as a

knowledge representation tool. VPA is used in the knowledge acquisition process

of KBDS because it is a powerful tool extracting knowledge from human experts

when they solve a real managerial decision problem with their heuristic knowledge

combined with decision-analytic knowledge and domain-specific knowledge. VPA

makes it especially easy to transfer the extracted knowledge to a knowledge base

using a well-defined coding scheme. The influence diagram combined with MADM

is also used to make it easy to elicit knowledge of a complex decision problem and

to apply the knowledge to similar problems. To elicit more accurate probabilities of

an influence diagram, an expansion procedure is introduced. It includes a discussion

transferring knowledge between similar decision problems. Some guidelines are

suggested for the development of KBDS.

Defining a class and storing it efficiently for similar problems is a promising

research area. A CBR algorithm combined with defining a class is also a possible

area for further research.

REFERENC ES

Beck, H. W., T. Anwar, and S. B. Navathe. 1994. Conceptual clustering algorithm for database schema design.

IEEE Transactions on Knowledge and Data Engineering 6(3):396±411.

Chung, T. Y., J. K. Kim, and S. H. Kim. 1992. Building an influence diagram in a knowledge-based decision system.

Expert Systems with Applications 4:33±44.

Ericsson, K. A., and H. A. Simon. 1984. Protocol analysis: Verbal reports as data. Cambridge, Mass.: MIT Press.

Holtzman, S. 1985. Intelligent decision systems. Ph.D. thesis, Dept. of Engineering-Economic Systems. Stanford,

Calif.: Stanford University.

148 J.-K. Kim

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

6:40

08

Oct

ober

201

4

Page 21: Knowledge acquisition for knowledge based decision systems

Holtzman, S. 1989. Intelligent decision systems. Reading, Mass.: Addison-Wesley.

Howard, R. 1984. The evolution of decision analysis. In The principles and applications of decision analysis, vol.

I, eds. R. Howard and J. Matheson. Menlo Park, Calif.: Strategic Decision Group.

Howard, R. 1988. Decision analysis: Practice and promise. Management Science 34(6):679±695.

Howard, R., and J. Matheson. 1984. Influence diagrams. In The principles and applications of decision analysis,

vol. II, eds. R. Howard and J. Matheson. Menlo Park, Calif.: Strategic Decision Group.

Kim, J. K. 1991. A knowledge-based system for decision analysis. Ph.D. thesis, Dept. of Industrial Engineering,

KAIST, Seoul, Korea.

Kim, J. K. 1995. A study on the development of intelligent decision systems using influence diagram. Journal of

the Korean OR/MS Society 20(3):77±104.

Kim, J. K., T. Y. Chung, and S. H. Kim. 1992. KIDS: A knowledge-based decision system to build an influence

diagram. Paper presented at the First World Congress on Expert Systems, Orlando, Florida, 16±19 Dec.

Kim, J. K., B. S. Kang, and S. H. Kim. 1990. Decision-theoretic approach to a knowledge-based clinical system.

Expert Systems with Applications 1(1):3±15.

Kim, S. H., and J. K. Kim. 1991. Explanation in a decision-theoretic consulting system: An axiomatic approach.

Applied Artificial Intelligence 5:393±409.

Matzkevich, I., and B. Abramson. 1995. Decision analytic networks in artificial intelligence. Management Science

41(1):1±22.

McGovern, J., D. Samson, and A. Wirth. 1991. Knowledge acquisition for intelligent decision systems. Decision

Support Systems 7:263±272.

Merkhofer, M. W. 1990. Using influence diagrams in multi-attribute utility analysisÐ improving effectiveness

through improving communication. In Influence diagrams, belief nets and decision analysis, eds. R. M. Oliver

and J. Q. Smith, pp. 297±319. New York: John Wiley.

Merkhofer, M. W., and R. L. Keeney. 1987. Multi-attribute utility analysis of alternative sites for the disposal of

nuclear waste. Risk Analysis 7:173±194.

Miller, A. C., M. W. Merkhofer, R. A. Howard, J. B. Matheson, and T. R. Rice. 1976. Development of automated

computer aids for decision analysis. SRI International Technical Report, Menlo Park, Calif.

Moore, E. A., and A. M. Agogino. 1987. INFORM: An architecture for expert-directed knowledge acquisition.

International Journal of Man-Machine Studies 26:213±230.

Newell, A., and H. A. Simon. 1977. Human problem solving. Englewood Cliffs, N.J.: Prentice-Hall.

Olmsted, S. M. 1983. On representing and solving decision problems. Ph.D. thesis, Dept. of Engineering-Economic

Systems, Stanford University, Stanford, Calif.

Owen, D. L. 1984. The use of influence diagrams in structuring complex decision problems. In The principles and

applications of decision analysis, vol. I, eds. R. Howard and J. Matheson. Menlo Park, Calif.: Strategic

Decision Group.

Park, K. S., S. H. Kim, and J. K. Kim. 1993. An artificial neural network approach to the decision class analysis.

In 93 Korean/Japan joint conference on expert systems, Seoul, Feb. 2 ± 5, pp. 717±729.

Reed, J. 1989. Building decision models that modify decision systems. Paper presented at the Thirteenth Annual

Symposium on Computer Applications in Medical Care (SCAMC).

Rege, A., and A. M. Agogino. 1988. Topological framework for representing and solving probabilistic inference

problems in expert systems. IEEE Transactions on Systems, Man, and Cybernetics 18(3):402±414.

Russel, S., S. Sampath, and A. Agogino. 1988. Creating influence diagrams from examples. Berkeley Expert

Systems Laboratory, University of California, Berkeley.

Shachter, R. D. 1986. Evaluating influence diagrams. Operations Research 34(6):871±882.

Shachter, R. D. 1988. Probabilistic inference and influence diagram. Operations Research 36(4):589±604.

Smith, G. F. 1988. Towards a heuristic theory of problem structuring. Management Science 34(12):1489±1506.

Smith, G. F. 1993. Defining real world problems: A conceptual language. IEEE Transactions on Systems, Man,

and Cybernetics 23(5):1220±1234.

Knowledge Acquisition for Knowledge-Based Decision Systems 149

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

6:40

08

Oct

ober

201

4

Page 22: Knowledge acquisition for knowledge based decision systems

PAGE 150 BLANK

Dow

nloa

ded

by [

Uni

vers

ity o

f C

onne

ctic

ut]

at 0

6:40

08

Oct

ober

201

4