A strategy for determining the optimal domain for knowledge based decision support systems

342 European Journal of Operational Research 48 (1990) 342-350 North-Holland

Theory and Methodology

A strategy for determining the optimal domain for knowledge based decision support systems

James R. Marsden Department of Decision Science and Information Systems, University of Kentucky, Lexington, K Y 40506-0034, USA

David E. Pingry University of Arizona, USA

Robert D. St. Louis Arizona State University, USA

Abstract: The objective of a knowledge based decision support system (KBDSS) is to enhance the ability of decision makers to respond to opportunities/problems. The set of tasks for which the KBDSS can provide some assistance is called the domain of the KBDSS. The set of tools and rules which the KBDSS uses to help with those tasks is called the problem processing system (PPS) of the KBDSS. This paper presents a procedure for determining the optimal domain-PPS combination for a KBDSS. A specific application is provided to illustrate why the procedure is needed and how it works.

Keywords: Information, decision, decision support systems, comRuters

I. Introduction

The objective of a knowledge based decision support system (KBDSS) is to enhance the ability of decision makers to respond to opportunities/ problems. A KBDSS may encompass elements of a management information system (MIS), a model management system (MMS), a decision support system (DSS), or an expert system (ES).

High employee turnover creates a situation within which a KBDSS may be very useful. Every time an employee leaves a position, regardless of whether the employee is promoted, transferred,

Received September 1988; revised May 1989

retired or terminated, the company loses position- specific knowledge that can be reacquired only through experience or specific training. A KBDSS can be used to store position-specific knowledge and thereby reduce the time and effort required to train employees given new task assignments. Moreover, if appropriately designed, the KBDSS also can be used to improve the job performance of employees whose job assignments have not changed.

Two obvious KBDSS design questions are: (1) What tasks should the KBDSS help em-

ployees perform? (2) What tools and rules should be embodied

in the KBDSS in order to help employees perform the selected tasks?

0377-2217/90/$03.50 © 1990 - Elsevier Science Publishers B.V. (North-Holland)

J.R. Marsden et al. / Determining the optimal domain for KBDSSs 343

The set of tasks or problems for which the KBDSS can provide some assistance is called the domain of the KBDSS. The set of tools and rules which the KBDSS uses to help with those tasks is called the problem processing system (PPS) of the KBDSS.

Determining the optimal domain for a KBDSS in a realistic situation is, by any criteria, a complex problem. The complexity is caused by the large number of domain-PPS combinations that are possible for any realistic problem, and by the difficulty of associating benefits and costs with each of the domain-PPS combinations. This complexity often is articulated in the literature by saying the problem is unstructured a n d / o r the benefits and costs are intangible. As a conse- quence, alternative domain-PPS combinations generally are evaluated using ad hoc search strategies.

A recognized problem with ad hoc search strategies is that they typically ignore synergistic benefits and joint cost savings (see discussions in Marsden and Pingry, 1986; and Ahituv and Halpern, 1982). The objective of this paper is to present and illustrate a strategy for searching the set of feasible KBDSS domain-PPS combinations. The strategy is designed to increase the probability that the optimal (profit maximizing) domain- PPS combination is in the set of alternatives that are evaluated for contribution to net profits. The strategy is illustrated by applying it to an employee turnover situation confronted by the Gar- rett Engine Division of the Allied-Signal Aero- space Company (an Arizona manufacturing company). For this example, the performance of our strategy is compared to that of some standard ad hoc procedures.

2. Developing a search strategy

Most of the terms used in this paper follow common usage. However, before developing our search strategy, specific definitions are provided for the four most important terms.

Knowledge Based Decision Support System (KBDSS) - A system designed to enhance decision task performance through the use of data, models or rules.

K B D S S Domain The set of decision tasks for

which the KBDSS is designed to enhance performance.

K B D S S Problem Processing System (PPS) - The set of tools and rules which the KBDSS uses to improve decision task performance (data base management system, model management system, inference engine, model structures, solution al- gorithms, rules, etc.).

K B D S S Range - The set of objective function values (e.g., contributions to company net profits) associated with the set of feasible KBDSS domain-PPS combinations.

These definitions cover the full range of KBDSS possibilities, from the narrowest data base management system to the most complex expert system. The KBDSS concept is broadly defined in order to emphasize the generality of the proposed search strategy. That is, regardless of the complexity of the KBDSS, the optimal domain can be determined only if a mapping exists from each potentially useful domain-PPS combination to the range of the KBDSS. Using the terminology defined in this section, the issues to be discussed in the rest of this paper are:

1. Is it possible to construct the required mapping?

2. Can the funct ion/mapping be optimized? 3. Can a practical strategy for constructing and

optimizing the funct ion/mapping be developed? 4. How does the performance of the proposed

strategy compare with the performance of well known ad hoc procedures?

The first two questions are equivalent to ask- ing: Can the problem of evaluating alternative KBDSS domain-PPS combinations be structured in such a way that a meaningful optimal solution may be determined? Though the terms problem structure or problem unstructure appear fre- quently in the literature, their definitions are not precise (see Simon, 1960, 1972; Keen and Scott- Morton, 1978; Moore and Chang, 1985; Landry et al., 1985). When we say that a problem is structured, we mean that the problem specification satisfies the following three conditions (see Mars- den and Pingry, 1987, 1990):

(1) The states, actions, and goals of the problem (and the relationships between these elements) are known.

(2) The decision model representation is adequate for the intended application.

344 J.R. Marsden et al. / Determining the optimal domain for KBDSSs

(3) The decision model representation is solva- ble.

If any of the above conditions is not satisfied, then the problem, or at least its current representation, is unstructured. In general, saying the KBDSS evaluation problem is unstructured or saying the benefits or costs of the KBDSS are intangible, is equivalent to saying one of the above conditions is not satisfied. Such statements generally precede the adoption of ad hoc evaluation procedures.

This paper contends that the three conditions for structure usually can be satisfied for the KBDSS evaluation problem. All that is required are estimates for the benefits and costs of a finite set of domain-PPS combinations which are repre- sentative of the choices available. In practice, the contention that the three conditions for structure usually can be satisfied is equivalent to the contention that the evaluation of alternatives is often worth the trouble (the cost in time and resources is exceeded in value by the improved solution).

Given that the KBDSS evaluation problem is structured as described above, the answers to the third and fourth questions hinge on whether or not a strategy can be designed for generating a finite set of domain-PPS combinations which is identifiably superior to the set generated by typi- cal ad hoc strategies. The argument presented in this paper suggests that such a strategy exists for KBDSS evaluation. We base our position on the following two observations:

1. KBDSS systems are rife with potential joint cost savings and synergistic benefits.

2. Most ad hoc evaluation procedures do not take account of joint cost savings and synergistic benefits.

The implications of these observations for a search strategy are:

1. The search strategy should be broad enough to capture cost and benefit interactions.

2. The search strategy should allow for over- laps in KBDSS domain-PPS combinations.

To illustrate these observations, consider the following simple example. Suppose the potential domain of a KBDSS consists of four tasks, and the potential PPS consists of three tools. In this case there are 105 possible domain-PPS combinations (15 task combinations times 7 tool combinations). Although many of the possible domain-pPS combinations can be discarded immediately be-

cause the tools do not correspond to the tasks, the set of domain-PPS combinations for which the tools do correspond to the tasks generally is too large to permit an evaluation of the benefits and costs for each combination. A search strategy nar- rows the set of combinations for which benefits and costs are to be generated. A major point of this paper is that ad hoc rules inappropriately narrow the set.

For example, one possible ad hoc approach to KBDSS development is to perform a benefit cost analysis each time a decision must be made concerning whether to initiate or enhance a KBDSS and to let the results of that analysis determine whether a particular task is added to the domain and whether a particular data set, model or solver is added to the PPS. Another ad hoc strategy is to evaluate the benefits and costs of every possible tool that could be put in the PPS, and then select the set of tools which yields the highest present value. A third ad hoc strategy is to specify the broadest possible domain, evaluate the benefits and costs of each tool that could be used to help with the tasks in the domain, and then select the set of tools which yields the highest present value. But the total contribution of a KBDSS is the sum of its marginal contributions if and only if there are no joint cost savings or synergistic benefits. Hence the above ad hoc strategies can assure the selection of the optimal domain-PPS combination if and only if there are no joint cost savings or synergistic benefits.

Rarely, however, does a situation exist, espe- cially in the case of KBDSS development, in which there are no joint cost savings or synergistic benefits. Therefore the search strategy for narrowing the set of domain-PPS combinations should not assume them away.

The strategy proposed in this paper for determining the optimal scope-domain combination is to look for joint costs or joint benefits and to explicitly account for the most significant ones. This makes it possible to group domain-PPS combinations into nearly independent subgroups (few joint costs or joint benefits between subgroups), and greatly reduces the number of domain-PPS combinations that must be evaluated. The strategy can be broken down into the following steps:

1. Identify the set of potential decision tasks (i.e. determine the potential domain of the KBDSS).

J.R. Marsden et al. / Determining the optimal domain for KBDSSs 345

2. Identify the skills required to perform the set of potential decision tasks.

3. Identify the tools and rules which may improve performance for the set of potential decision tasks (i.e. determine the potential components of the PPS).

4. Identify a reasonable set of domain-PPS combinations (i.e. identify the tools which can help with each task).

5. Select a subset of the domain-PPS combinations identified in step 4 by examining joint costs and synergistic benefits.

6. Determine the net profit contribution associated with each domain-PPS combination selected in step 5.

7. Identify the best (profit maximizing) domain-PPS combination.

The advantage of the proposed strategy over the usual ad hoc strategies is that it takes account of joint cost savings and synergistic benefits, which almost certainly exist. The disadvantage is a some- what more complex process for selecting the set of domain-PPS combinations to be evaluated. Be- cause the same number of domain-PPS combinations may be selected for evaluation, because it is relatively inexpensive to select domain-PPS combinations for evaluation, and because it can be very expensive to evaluate domain-PPS combinations (calculate the benefits and costs associated with the selected domain-PPS combination), the preferred search strategy is the one which maxi- mizes the probability that the set of evaluated domain-PPS combinations includes the optimal combination. How to group domain-PPS combinations into nearly independent subgroups, and how to determine payoffs for the remaining domain-PPS combinations is illustrated via an example in the next section.

3. The Garrett example

To illustrate our strategy for determining the optimal domain-PPS combination for a KBDSS, we present an example concerning experience losses due to employee turnover. The data used are from work the authors did with an engineering group at the Garrett Engine Division of the Al- lied-Signal Aerospace Company, a large, turbine engine manufacturing company located in Phoe-

nix, AZ. This group has responsibility for qualifying outside vendors and testing internal products.

The members of this work group are primarily metallurgists by training. When hired, they typically do not have a sufficiently strong background in statistics and experimental design to satisfacto- rily complete the job tasks. That is, it is quite common for the company to hire metallurgists and expect significant on-the-job statistical learn- ing to occur. This reflects a decision by this company that it is easier to train a metallurgist in the requisite statistical tools than it is to train a statis- tician in the requisite metallurgical skills.

On-the-job training also is required to learn how to test the materials. The tests which need to be performed by this group tend to be very specialized, involve extremely expensive test equip- ment, and use expensive test specimens. Because of the specialized nature and costs of these tests, university education does not usually include the appropriate training.

On-the-job training may eventually provide an employee with the needed skills. But training peri- ods cost money and involve a redirection of manpower from production to training. Further, as has been the case for the company considered here, as employee turnover increases, the length of the 'payback period' for such training decreases, diminishing the benefi t-cost ratio of such training. The group leader recognized that one possibil- ity for lessening the impacts of employee turnover might be the development of a KBDSS which would serve as a trainer for new employees or for current employees. However, because he was not an expert on decision support systems, he did not know how to evaluate their potential to help with his problem. When he contacted the authors, his problem became an excellent opportunity for evaluating our proposed strategy.

In this example, KBDSS domain refers to the set of decision tasks addressed by the KBDSS; KBDSS PPS refers to the data bases, statistical analysis software, inference engine, etc. used by the KBDSS to enhance performance of the decision tasks; and KBDSS range refers to the impact of a selected KBDSS domain-PPS combination on company profit. To compute such values, it is necessary to determine expected savings in training costs (saved time as well as saved direct ex- penses) and expected increases in productivity due to improvements in training and the ability to


Table 1 Decision tasks

Qualify Vendors 1 (QV1) Compare test data to baseline data using techniques

such as regression analysis (with binary variables) and analysis of variance.

Consumes 30% of the group's time Qualify Vendors 2 (QV2)

Perform hypothesis tests on means, variances, and distributional forms.

Consumes 20% of the group's time Select Test Types and Test Specimen Configurations (ST1)

Determine the type of test to be run (e.g., tensile, creep, low cycle fatigue, etc.) and determine the physical requirements for the tests (e.g., three point or four point bending; electronic, microscopic, or ultrasonic inspection; drop or impact; etc.).

Consumes 25% of the group's time Plan Tests 1 (PT1)

Determine the number of temperature levels to use, the number of stress levels to use, the run out time, etc.

Consumes 10% of the group's time Plan Tests 2 (PT2)

Determine the number of observations required to get a good model.

Consumes 10% of the group's time Consulting (CON)

Giving general metallurgical or statistical help to other units within the company.

Consumes 5% of the group's time

a ny par t or all of the six task groups. The KBDSS doma in could, for example, be so l imited as to enhance only the compar i son of new data to base- l ine data using b inary variables, or so broad as to enhance all of the decision tasks in all six of the

categories. Ident i fy required skills. The second step in the

proposed search strategy is to ident i fy the skills required to perform the potent ia l decision tasks. Conversa t ions with Garre t t employees led to the classification of required skills as statistical or metallurgical, and the deve lopment of the specific

enumera t ion of skills shown in Table 2. Al though a more detailed classification and enumera t ion

certainly could have been developed, the skills listed in Table 2 were judged by Garre t t to be an adequate summary of the skills needed to qualify

vendors and test in terna l products. ldent i fy tools and rules. In this example, the set

of tools and rules which may be used to improve performance for the set of potent ia l decision tasks was substant ia l ly restricted as a result of prior company decisions. Garre t t current ly has licensed

IMSL ( In te rna t iona l Mathemat ica l and Statistical Library) and P R O L O G (Programming in Logic), and will no t allow its employees to use any other

set of statistical rout ines or logic based program-

consul t with the KBDSS. These benefits are, of course, offset by the cost of the KBDSS.

The costs of employee turnover and the costs of

a KBDSS designed to t ra in employees p robab ly

are as difficult to quant i fy as any costs that are incurred by a company. When the authors were contacted, Garre t t ' s group leader had absolutely no data on the cost to t rain employees, the extent to which a KBDSS could be used to t rain em-

ployees, or the cost to bui ld and m a i n t a i n a KBDSS. He s imply felt that a KBDSS might be helpful. If our proposed strategy for de te rmin ing the opt imal KBDSS domain-PPS combina t ion works for this problem, it p robab ly will work for a wide range of problems.

Ident i fy potent ia l decision tasks. The first step in the strategy is to ident ify the set of decision tasks which mus t be performed by the engineering group. Conversa t ions with members of the group revealed that they divide their work time among the six decision tasks shown in Table 1. A KBDSS could be developed to enhance the performance of

Table 2 Skills required for decision tasks

Statistical skills S1 General statistical skills (familiarity with probability distri-

butions, estimation, standard errors, types of confidence and tolerance intervals, hypothesis testing, type I errors, type II errors, power functions, etc.)

$2 Regression analyses (familiarity with general linear model tests, comparison tests using binary variables, simulta- neous inferences, transformations, influence measures, collinearity measures, comparison tests for variances, etc.)

$3 Tests on means, oariances and distributional forms (familiarity with analysis of variance tests, chi-square tests, Kolmogorov-Smirnoff tests, F-tests on variances, etc.)

$4 Experimental design (familiarity with the principles of experimental design, sample size determination, accep- tance sampling, sequential sampling, Mil. standards, OC curves, etc.)

Metallurgical skills M1 Testing (familiarity with the procedures for conducting

tests on tensile strength, ductility, low cycle fatigue, etc.) M2 Material properties (familiarity with material properties at

various temperature levels, load levels, operating ranges, etc.)

J.R. Marsden et al. / Determining the optimal domain for KBDSSs

Table 3 Decision task-PPS tool correspondence table

347

$1 $2 S3 $4 M1 M2 IMSL P R O L O G

QV1 X X X X QV2 x X X X ST1 X X PT1 X X PT2 X X X X CON X X X X X X X X

ming language. Since the company has decided to use IMSL and PROLOG, alternative statistical packages and programming languages can be ignored.

Given this restriction, there is no choice with respect to the tools in the PPS. The only choices are with respect to the knowledge bases or rules to help with each of the skills. In total there are 64 skill combinations, ranging from zero skills to all six of the skills shown in Table 2 (S1, $2, $3, $4, M1, M2). A KBDSS to enhance all six of these skills would have the broadest possible PPS. A KBDSS to enhance none of these skills would have the narrowest possible PPS (no problem processing system).

Identify a reasonable set of domain-PPS combinations. It makes no sense to include a task in the domain of a KBDSS unless some tool is included in the PPS of the KBDSS to enhance the performance of that decision task. A decision task-PPS tool correspondence table makes it possible to rule out nonsensical domain-PPS combinations. Table 3 is the decision task-PPS tool correspondence table for this example.

Select subset of domain-PPS combinations to evaluate. The fifth step, and the most crucial for the strategy outlined above, is to reduce the scope-domain combinations to a manageable number. The key to this reduction is finding nearly independent (few joint costs or synergistic benefits) groups of task- tool combinations. For this example, discussions between Garret t employees and the authors indicated that the statistical skills and the metallurgical skills are essentially independent. This reduced the problem to two problems with, respectively, 20 and 4 domain-PPS combinations. Ignoring the empty set and combinations for which there is no task- tool correspondence, led to the 10 domain-PPS combinations which are listed in Table 4. Note that alternatives

4 - 6 would not have been generated by any of the ad-hocstrategies discussed above.

Evaluate selected domain-PPS combinations. In this example, there are two major sources of benefits from the KBDSS. These are

(1) improved efficiency for all employees in the group; and

(2) reductions in training time for employees new to the group (fewer hours required on the part of both the trainer and the trainee).

The effect of improved efficiency is a direct function of the frequency of occurrence of the problems handled by the KBDSS. The percent of the group's time devoted to each of the decision tasks was given in Table 1. From Table 1, it is easy to find the percent of the group's time spent on the decision tasks in the domain of each of the KBDSSs shown in Table 4. The percents are: KBDSS1 = 31%, KBDSS2 = 21%, KBDSS3 = 11%, K B D S S 4 = 52%, KBDSS5 = 42%, KBDSS6 = 32%, K B D S S 7 = 63%, K B D S S 8 = 26%, KBDSS9 = 11%, KBDSS10 = 37%. (Note: Gar- rett 's estimated 'consulting 5%' has been rolled into the other decision tasks, 1% into each decision task.)

Table 4 Selected domain-PPS combinations

KBDSS PPS Domain

KBDSS1 S1, $2 QV1 KBDSS2 S1, $3 QV2 KBDSS3 S1, $4 PT2 KBDSS4 S1, $2, $3 QV1, QV2 KBDSS5 $1, $2, $4 QV1, PT2 KBDSS6 S1, $3, $4 QV2, PT2 KBDSS7 S1, $2, $3, $4 QV1, Qv2, PT2

KBDSS8 M1 ST1 KBDSS9 M2 PT1 KBDSS10 M1 M2 ST1, PT1

348

Table 5 Benefit-cost calculations a

J.R. Marsden et al. / Determining the optimal domain for KBDSSs

KBDSS Expected Maximum Weekly Training Employees Present Total % of training salary savings trained value training training weeks per emp. per year weight savings (1) (2) (3) (4) (5) (6) I, (7)

KBDSS1 0.76 4 500 1520 1 3.79 KBDSS2 0.76 4 500 1520 1 3.79 KBDSS3 0.76 3 500 1140 1 3.79 KBDSS4 0.76 6 500 2280 1 3.79 KBDSS5 0.76 5 500 1900 1 3.79 KBDSS6 0.76 5 500 1900 1 3.79 KBDSS7 0.76 7 500 2660 1 3.79

KBDSS8 0.65 3 500 975 1 3.79 KBDSS9 0.76 2 500 760 1 3.79 KBDSS10 0.69 5 500 1725 1 3.79

Percent of Effic. Effic sav. Number of Total effic. Devdp. Net c time in task improv, per employee employees savings costs benefits (8) (9) (10) (11) (12) (13) (14)

5762 5762 4321 8643 7202 7202

10083

3696 2881 6539

0.31 0.05 403 5 7638 7000 6400 0.21 0.05 273 5 5174 7000 3936 0.11 0.05 143 5 2710 6000 1031 0.52 0.05 676 5 12813 11000 10456 0.42 0.05 546 5 10349 11000 6551 0.32 0.05 416 5 7885 10000 5087 0.63 0.05 819 5 15523 14000 11606

0.26 0.05 338 5 6406 3000 7102 0.11 0.05 143 5 2710 3000 2592 0.37 0.05 481 5 9117 6000 9656

a Definitions: (4) = (1)*(2)*(3), (7) = (4) * (5) * (6), (10) = 52 * (3) * (8)* (9), (12) = (6)*(10).(11), (14) = (7) 4- (12) - (13).

b Assumes the benefits are discounted over 5 years using a rate of 10%. c For KBDSS domains which include more than one decision task, the net benefit calculations take into account joint cost savings.

They are not the sum of the net benefits for the individual decision tasks. i

I n a d d i t i o n to i n f o r m a t i o n o n the p e r c e n t o f

t he g r o u p ' s t i m e s p e n t o n the d e c i s i o n t a sks in t he

d o m a i n o f t he K B D S S , d e t e r m i n i n g the s av ings

f r o m i m p r o v e d e f f i c i e n c y r e q u i r e s i n f o r m a t i o n o n

the n u m b e r o f p e r s o n s in the g r o u p , t he ave r age

w e e k l y sa l a ry o f t h e p e r s o n s in the g roup , t he

p e r c e n t i m p r o v e m e n t in e f f i c i ency , a n d the r a t e a t

w h i c h the s av ings a re to b e d i s c o u n t e d ove r t ime .

D i s c u s s i o n s w i t h G a r r e t t l ed us to use t he fo l low-

ing values : a g r o u p size o f 5, an ave r age w e e k l y

sa l a ry o f $500, a 5% i m p r o v e m e n t in e f f i c iency ,

a n d a n i n t e r n a l r a t e o f r e t u r n o f 10% d i s c o u n t e d

o v e r a f ive y e a r p e r i o d .

I t m a y a p p e a r t h a t T a b l e 5 a l so a s s u m e s all

m e m b e r s o f the g r o u p a re i d e n t i c a l in t e r m s o f t he

Table 6

1. Statistical skills needed for qualifying vendors No deficiency: 5% Slight deficiency: 10% Moderate deficiency: 35% Severe deficiency: 50%

2. Metallurgical skills needed for selecting test types No deficiency: 5% Slight deficiency: 20% Moderate deficiency: 50% Severe deficiency: 25%

3. Statistical and metallurgical skills needed to plan tests No deficiency: 5% Slight deficiency: 10% Moderate deficiency: 35~ Severe deficiency: 50%

J.R. Marsden et aL / Determining the optimal domain for KBDSSs 349

tasks undertaken, or all members of the group have identical skills. In fact, however, we assume that all members of the group are equally likely to leave the group. During the eight years that the authors have worked with Garrett, every member of the group (including the group leader) has left and been replaced.

The reduction in training time is a function of the amount of training required, and the extent to which the KBDSS can substitute for a human trainer a n d / o r reduce the time it takes to train an employee. After much discussion between the authors and the staff at Garrett, the probabilities in Table 6 were estimated for the skill shortcom- ings of the new employees to the group. Based upon Garrett 's estimate that a slight deficiency requires one third of the training, a moderate deficiency requires two-thirds of the training, and a severe deficiency requires all of the training, the average percent of the full training that will be needed for each of the decision tasks is: QV1 = 0.76, QV2 = 0.76, ST1 = 0.65, PT1 = 0.76, PT2 = 0.76.

The KBDSS cannot eliminate all of the one-on- one training required of employees new to the group. It also may not be more effective than a human trainer. Overall for a severe deficiency, it was estimated that the KBDSS could save the following amounts of trainers' times for each of the skills listed: S1 = 2 weeks, $2 = 2 weeks, $3 = 2 weeks, $4 --- 1 week, M1 = 3 weeks, M2 = 2 weeks. The savings from reduced training are calculated in Table 5. The calculations assume one new employee enters the group each year.

Select best d o m a i n - P P S combinat ions. Inspec- tion shows that for this example, the optimal domain-PPS combination is

domain = (QV1, QV2, ST1, PT1, PT2)

and

PPS = (S1, $2, $3, $4, M1, M2) .

Table 7

Domain KBDSS Net benefits

QV1 KBDSS1 $3 345 QV2 KBDSS2 1866 PT2 KBDSS3 - 53 QV1, QV2 KBDSS4 5 331 QV1, PT2 KBDSS5 2 411 QV2, PT2 KBDSS6 1933 QV1, QV2, PT2 KBDSS7 5 397

ST1 KBDSS8 4540 PT1 KBDSS9 1507 ST1, PT1 KBDSS10 6 009

same solution as our strategy. This would have occurred because the net benefit is positive for each KBDSS that supports a single decision task - KBDSSs 1, 2, 3, 8 and 9. Total net benefits, however, would have been underestimated by $202 ($11 606 + $9656 = $21 262 vs. $6400 + $3936 + $1031 + $7102 + $2 591 = $21060).

Moreover, if the efficiency improvement had been 3% rather than 5%, the net benefits would have been as shown in Table 7. In this case an ad-hoc search strategy which assumes there are no joint cost savings or synergistic benefits would not have included PT2 in the domain of the KBDSS since its net benefit is a negative $53, and the net loss to the company would have been $66. The ad hoc procedure also would have incorrectly estimated the net benefits for the

domain = { QV1, QV2, PT1, PT2) ,

problem processing system

= ( $ 1 , $2, $3, $4, M1, M2}

combination as $11 258, when it actually equals $11 340. In fact, the ad-hoc procedure will cor- rectly estimate the net benefits for a KBDSS domain-PPS combination only if there are no joint cost savings or synergistic benefits among the decision tasks in the domain of the KBDSS.

This selection is based on the largest net return value in each of the two independent searches. KBDSS7 yields the highest net return in the first group ($11606), while KBDSS10 yields the highest net return in the second group ($9 656).

For this data, the use of an ad hoc search strategy that assumes there are no joint cost savings or synergistic benefits would have yielded the

4. S u m m a r y a n d c o n c l u s i o n

This paper presents a seven step strategy for determining the profit maximizing domain-PPS combination for a knowledge based decision support system. A major tenet of the strategy is that KBDSS systems are rife with potential joint cost


savings and synergis t ic benefi ts , and that these in te rac t ions should be cons idered when a t t empt - ing to es t imate the benef i ts and costs of a KBDSS. A t the same time, however, we have taken care to po in t out that as p r o b l e m complex i ty grows, de- te rmining all in te rac t ion mapp ings can become proh ib i t ive ly expensive. The s t ra tegy suggested in this p a p e r a t t empt s to avoid the undes i rab le out- comes re la ted to ignor ing in te rac t ions while a t the same t ime avoid ing the high costs associa ted with ful ly de ta i l ing all poss ib le in teract ions . To accom- pl ish these ends, we suggest g roup ing decis ion tasks and p r o b l e m process ing tools in to a rela- t ively small n u m b e r of near ly i ndependen t categories, and then cons t ruc t ing fair ly rough esti- ma tes of the payof f s associa ted with each of the grea t ly reduced n u m b e r of doma in -PPS combina - t ions. Sensi t ivi ty analysis can then be used to de t e rmine which d o m a i n - P P S combina t ions should be chosen for more precise es t imat ion and evaluation.

The s t ra tegy is i l lus t ra ted using an employee tu rnover example . The costs of employee tu rnover and the costs of a K B D S S des igned to t ra in em- p loyees are very diff icul t to quant i fy . W h e n the au thors were contac ted , G a r r e t t ' s group l eader had abso lu te ly no da t a on the cost to t ra in employees , the extent to which a K B D S S could be used to t ra in employees , or the cost to bu i ld and ma in t a in a KBDSS. H e s imply felt that a K B D S S might be helpful . The fact tha t our s t ra tegy for de t e rmin ing the op t ima l K B D S S d o m a i n - P P S c o m b i n a t i o n worked for this very complex p rob - lem encourages us to conjec ture that our s t ra tegy could yield s ignif icant benef i ts for a wide range of p r o b l e m types.

Acknowledgement

The au thors wish to thank K a t h y Clark and Vicki Panhuise of G a r r e t t Engine Divis ion and N a n u Menon, fo rmer ly of the same company , for their coopera t ion and input . A n y mis takes or mis-

i n t e rp re t a t i ons are the r e spons ib i l i t y of the authors .

References

Ackoff, R. (1967), "Management misinformation systems", Management Science 14 (4), B147-B156.

Ahituv, N., and Halpern, J. (1982), "Data and reports: Con- tents design and users allocation", The Journal of Systems and Software 2, 193-199.

Ahn, T., and Grudnitski, G. (1985), "Conceptual perspectives on key factors in DSS development: A systems approach", Journal of Management Information Systems 2 (1), 18-32.

Dolk, D., and Konsynski, B. (1984), "Knowledge representation for SP-model management systems", IEEF Transac- tions or Software Engineering SE-10 (6), 619-628.

Gerrity, T.P. (1971), "The design of man-machine decision systems: An application to portfolio management", Sloan Management Review 12 (2), 59-75.

Gremillion, L.L., and Pyburn, P.J. (1985), "Justifying decision support and office automation systems", Journal of Man- agement Information Systems 2 (1), 5-17.

Keen, P.G., and Scott-Morton, M. (1978), Decision Support Systems: An Organizational Perspective, Addison-Wesley, Reading, MA.

Klein, G., Konsynski, B., and Beck, P. (1985), "A linear representation for model management in a DDS", Journal of Management Information Systems 2 (2), 40-54.

Landry, M., Pascot, D., and Briolat, D. (1985), "Can DDS evolve without changing our view of the concept of 'problem'?", Decision Support Systems 9, 137-151.

Marsden, J.R., and Pingry, D.E. (1990), "A theory of decision support system portfolio design and evaluation", forthcom- ing in Decision Support Systems.

Marsden, J.R., and Pingry, D.E. (1986), "Generating an optimal information system: PMAX-SDLC and the redirection of MIS research", Journal of Management Information Sys- tems 3 (1), 32-51.

Moore, J.H., and Chang, M.G. (1980), "Design of decision support systems", Proceedings of the Hawaii International Conference on Systems Sciences, 1-8.

Simon, H.A. (1960), The New Science of Management, Harper and Row, New York.

Simon, H.A. (1972), "Theories of bounded rationality", in: Decision and Organization, C.B. McGuire and Roy Radner (eds.), North-Holland, Amsterdam.

Zwass, V. (1984), "Management and information systems-- Beyond the current paradigm", Journal of Management Information Systems 1 (1), 1-10.

Documents

A strategy for determining the optimal domain for knowledge based decision support systems