Upload
phamkiet
View
217
Download
0
Embed Size (px)
Citation preview
Brugha, C. (1998), "Structuring and Weighting Criteria in Multi Criteria Decision
Making (MCDM)", Trends in Multicriteria Decision Making: Proceedings of the
13th International Conference on Multiple Criteria Decision Making, Stewart, T.
J. and Van den Honert, R.C. (eds.): Springer-Verlag, p. 229-242. [MCDA]
Structuring and Weighting Criteria in Multi Criteria
Decision Making (MCDM)
Cathal M. Brugha, University College Dublin.
Abstract: The implications of qualitative distinctions between multiple criteria
are considered. Some contributions to theory about the Analytical Hierarchy
Process (AHP) are challenged. Experiments on alternative criteria structures are
reported. These suggest that confusing structures are bad, but good structures are
better than none. Guidelines on how to develop a structure are given for a well
known case of the purchase of a house. It is suggested that differences between
decision alternatives should provide a first phase basis for discovering criteria. A
criteria tree should be structured ‘top down’ as a second phase by clustering
criteria on the basis of qualitative difference. On any level the differences
between criteria should follow relatively simple patterns. The rules used suggest
the relevance of work on the structure of qualitative decision-making which is
determined by Nomology, the science of the laws of the mind. Implications are
considered for weighting trade-offs between homogeneous clusters of criteria.
This should be done as a later ‘bottom up’ phase. The AHP scoring system is
challenged. Some tests of alternative scoring methods are reported.
Keywords. Analytic Hierarchy Process, Decision theory, Nomology, Qualitative
structuring
1. Introduction
This article develops from work by the author on the structure of qualitative
decision-making (Brugha 1998a, 1998b and 1998c) which is determined by
Nomology (Hamilton, 1877), the science of the laws of the mind. It provides a
basis for modelling the way people might differentiate multiple criteria. Also
relevant to this paper is psychophysics (Gescheider, 1985) which has provided an
empirical basis for the comparison of objects by means of relative measurement,
and is the basis of the Analytical Hierarchy Process (AHP) (Saaty, 1996).
The AHP has been the subject of much controversy, particularly to do with the
question of it apparently being the cause of rank reversal in some circumstances.
One problem that has concerned this author is the use by the AHP of the Right
Eigenvector method to synthesise multiple scores of relatively measured objects.
Crawford and Williams (1985), Barzilai et al (1987, 1992, 1994), Holder (1990)
and Lootsma (1993, 1996) have considered this at length. Lootsma’s proposed
variation, Geometric AHP, is now widely accepted and so this issue will not be
considered here. Two other issues which have concerned this author will be
considered in this article. The first is what this author sees as the inadequate
treatment of qualitative difference between criteria particularly in some
applications of the AHP. There seems to be some fundamental confusion about
what is meant by having multiple criteria. In the next section we illustrate some of
these confusions. We then review some criteria structuring experiments which
were carried out in University College Dublin. We then propose some guidelines
for structuring criteria using a well-known case for illustration.
The other issue considered here is the calculation of the weights of relative
importance of criteria and, in particular, the 1 to 9 semantic differential scoring
system used in the AHP (Saaty, 1980, 1990a). A semantic differential is used to
get a consensus on agreement about the importance of one item compared to
another, or in the AHP case agreement on how different two things are from each
other. This author has been concerned about this strength-of-agreement figure
being used to score the actual weight of that relative importance. Some
suggestions about the calculation of weights are given. Also some tests of
alternative weighting systems are reported, leading to some further guidelines.
2. Criteria Structuring
The AHP grew out of a need to accommodate qualitative differences between
criteria. Some difficulties with AHP have focused on the quantitative issue of
rank reversal. Saaty (1994), for example, introduced a new form of normalisation,
the ‘Ideal’ form, to avoid rank reversal being caused by the entry of irrelevant
alternatives or numerous copies of one alternative. He also used an argument
about the introduction of copies of a hat, for example, to justify the change in rank
of existing alternatives caused by the AHP model (Saaty, 1994). For instance, if
A is preferred to B and B is preferred to C, possibly the introduction of multiple
copies of C could lead to B being preferred to A, hence rank reversal.
There seems to have been a confusion here about the interaction that occurs
between alternatives and criteria. The above problem arose only indirectly
because of the multiple copies. Clearly the introduction into the mind of the
decision maker of the possibility of copies revealed a previously hidden criterion
based on the issue of uniqueness. We would suggest that, if the other criteria have
been modelled correctly, such a new criterion should be easy to fit into the
existing structure, and the weighting of that issue should be possible to do as an
extension of the existing system of comparisons of criteria.
In general, criteria should be included in a goal modelling process on the basis
of their relevance to choices between the existing alternatives. This can mean that
new criteria can be included or old ones dropped as new alternatives are
considered or excluded. The process is essentially about the interaction between
objectives and alternatives, i.e. between criteria and attributes. The proper
structuring of criteria facilitates their change as the necessity arises.
2.1 Car choice example
Schoner and Wedley (1989) have proposed ‘Referenced AHP’ and ‘B-G modified
AHP’ to deal with quantitative differences in attributes such as in the case of the
purchase of a car. The attributes which are relevant to the decision to choose
which car to buy were related quantitatively. From this author’s point of view
there was only one criterion, cost. Consequently the AHP should not have been
used. The car was expected to have a life of five years. Hence five years
maintenance was factored in with the price. The cars were expected to do 10,000
miles each year at an estimated cost of $1.50 per gallon. This was also factored
in. Each car had a different price, maintenance cost and fuel usage in gallons per
mile. For any car it is easy to calculate its estimated cost over the five years.
Thus, for any two cars it is easy to calculate their relative costs directly without
reference to the other cars. Applying the AHP in the usual way leads to rank
reversal when a new car alternative is introduced. ‘Referenced AHP’ reinstates
the correct relative relationship between alternatives. Before the new car was
introduced the following were the alternative cars and their attributes:
Car Price ($) Maintenance ($/yr.) Fuel (gal./mi.)
1 14,000 2,000 0.05
2 5,000 4,000 0.03
3 6,000 4,000 0.05
Total 25,000 10,000 0.13
The scale factors are q1 = 1 for price, q2 = 5 for maintenance and q3 = 10,000 × 5 ×
1.5 = 75,000 for fuel costs over the five years. The ratio of cost of car 1 to car 2 is
R1 214 000 5 2 000 75 000 055 000 5 4 000 75 000 03
27 75027
1 018,, , , ., , , .
,,250
.= + ∗ + ∗+ ∗ + ∗
= =
The vectors of local priorities were produced using the AHP procedure, but
clearly can be viewed from the data. The three cars score as follows on price,
maintenance and fuel:
1425
525
625
15
25
25
513
313
513
�
�
���
�
�
���
�
�
���
�
�
���
�
�
���
�
�
���
The vectors of the weights of importance of the three attributes are calculated
similarly using the AHP and can be seen to be derived from the total costs from all
three attributes over all three cars: $25,000 plus 5 × $10,000 plus $75,000 × .13
equals $84,750. Proportionately these are 0.295, 0.590 and 0.115. In AHP these
weights should remain the same. So, when a new car is included and these
weights are used a conflict can lead to a rank reversal between cars 1 and 2.
‘Referenced AHP’ prevents this conflict happening by restoring the true weights.
The strength of an AHP type procedure is that it can be used to synthesise scores
on qualitatively distinct criteria which might be measured most easily in terms
relative to one another. Not only was there no need to apply the somewhat
complicated AHP procedure to measure a relationship that was already known
quantitatively, it was not appropriate to apply it in this situation because all three
attributes were based on cost. The implication that the AHP procedure is usable in
every multi-attribute situation is more than just wrong, it distracts from its
contribution as a method for synthesising criteria whose multiplicity arises in the
mind of the decision maker.
‘Referenced AHP’ produces scaling factors that convert quantitatively related
attribute scores from AHP tables back into a common score. Because the AHP
should not be used to synthesise qualitatively similar attributes we believe there is
no need to use Schoner and Wedley’s (1989) proposed ‘Referenced AHP’ or ‘B-G
modified AHP’ which they adapted from Belton and Gear (1983, 1985).
2.2 Farmer’s field example
Schenkerman (1994) shows similar difficulties when the AHP is applied to a field
where the goal is the perimeter and the attributes are the distances on the
rectangular sides North-South (N) and East-West (E).
Unnormalised Normalised
Field N E Perimeter Priority N E Synthesised
A 100 800 1800 0.450 15
815
0.367
B 200 100 600 0.150 25
115
0.233
C 200 600 1600 0.400 25
615
0.400
Total 500 1500 4000 1.000 55
1515
1.000
Prioritisation on the basis of the perimeter using the AHP puts field A first.
Synthesisation using AHP normalisation puts field C first. Schenkerman shows
that the normalisation method causes a difficulty with absolute measurement (such
as this case), and that Referenced AHP, B-G modified AHP, the linking-pin
method and the supermatrix method simply undo the eigenvector normalisation.
In this case, the N and E attributes are brought back to the same dimension by re-
introducing weights of 5 and 15. The resolution to this problem is to recognise
that lengths and breadths of fields are not qualitatively distinct criteria for any
decision maker’s goal. Consequently the conflict implied by the illustration does
not apply.
2.3 Experiments With Alternative Trees
As part of their course assignments some alternative trees were tested by several
groups of final year students of the Bachelor of Commerce Degree in University
College Dublin. The decisions modelled mainly had to do with the choice they
had to make about what to do the following year. Part of their task was to
compare Naive AHP with Structured AHP, and then to compare both with
SMART. Naive AHP corresponds to the usual approach as in Saaty’s (1990b)
house example (below), which means pairwise comparisons of each criterion with
every other criteria. Structured AHP meant clustering the criteria into a hierarchy
based on qualitative similarities.
One group carried out the process with final year students of engineering on the
issue "What to do next year?" after graduating. In the first phase they identified
the following alternative activities the graduating engineers would consider: get a
job (Job), do a post-graduate course in Ireland (P/G Here), do a post-graduate
course abroad (P/G Abroad), or take a year off (Bum). They also identified their
main objectives as make money (Cash), develop work experience (Work Exp), get
better qualifications (B/Q), an issue to do with pressure and free time (Pres F/T),
and one to do with personal development (Per Dev). They then applied Naive
AHP to those five issues with one group of eight students, and Structured AHP to
another group of eight students, both randomly chosen and tested individually.
The latter was structured with Cash, Work Exp and B/Q clustered together and
called Economic (Figure 1). This author suggested that comparisons on any level
should be between things of a corresponding level of generality. Consequently
PRES F/T was called Personal Feelings and Per Dev was called Personal Growth.
The AHP package Expert Choice was used to synthesise the results. The most
striking outcome was that Pres F/T had a global weight of 0.128 using Naive AHP
for one group and 0.474 using Structured AHP under the heading of Personal
Feelings for the other group. It was felt that there may have been some other sub-
criteria under Personal Feelings which had been left out. Hence a possible benefit
of the structured approach would be that general categories such as Personal
Feelings might subsume other important but unarticulated feelings. The overall
inconsistency score was 0.19 for the naive approach and 0.07 for the structured
approach. The better score for the latter may have been partly due to having fewer
comparisons, but was more likely due to the structuring.
What To Do Next Year? Economic Personal Personal Feelings Growth Cash Work Exp B/Q Pres F/T Per Dev
Figure 1. Graduating Engineers' Choices
The following is a summary of the comments from all the groups. The Naive
AHP caused a problem when there were many criteria. This led to an excessive
number of questions, a drop off in interest towards the end of the questioning
process and higher inconsistencies. The Structured AHP led to higher
consistency scores. The grouping of criteria helped respondents to become aware
of criteria or attributes that they might have forgotten. The questions were more
specific and more easily understood. However, if the structuring was poorly done
there was a danger of confusion. It required a greater understanding of the issues
in order to do the structuring. Also, the added levels might lead to distortions in
the global weightings.
2.3.1 Comparison of Three Criteria Tree Structures
In a subsequent running of the course the above test was extended to a comparison
of three alternative structures. Several class groups were given the following task
specification. In the first phase of the study they had to find a homogeneous group
of similar respondents who had a real choice to make, interview them, determine
their three most preferred alternatives, and their four most important criteria. The
rest of this section describes one of the groups which based its project on the
choice a final year college student faces in the year following their departure from
college. The first phase determined the three alternatives to be Employment,
Travel and Postgraduate Study. The criteria were Income, Challenge, Education
and Experience. Three alternative trees were then constructed. The first tree was
comprised of the four criteria in Naive AHP form. Each of the respondents was
asked to group the four criteria into two pairs based on which were closest. Some
formed clusters of Income / Education and Challenge / Experience, others had
clusters of Income / Challenge and Education / Experience. This provided the
second tree. Each was then asked was there any criterion in one of the pairs which
seemed closer to the combined other pair. This provided the basis of the third
tree, as in Figure 2.
Criterion Tree (i) Income Challenge Education Experience .283 .314 .237 .166 Criterion Tree (ii) .767 1.304 .894 1.118 1.304 .767 Income Challenge Education Experience .190 .237 .361 .212 Criterion Tree (iii) 1.360 .735 1.140 .877 1.304 .767 Income Challenge Education Experience .329 .236 .274 .161
Figure 2. Alternative Criteria Trees
The respondents were asked to rate the different trees, to comment on the
process, particularly the ease of understanding it, and to evaluate the exercise. Of
the nine respondents in this case only one liked the first tree, two liked the second,
and six liked the third. Most of this group isolated Income as the most different as
in Criterion Tree (iii). These results fit some theoretical ideas developed by this
author from his work in applying Nomology to MCDM, which are discussed in
the next section.
The numbers in Figure 2 come from one of the respondents in this group.
Geometric AHP was used to synthesise the data. The below the line figures are
normalised to total one so as to allow comparison. The figures on the branches of
the trees multiply to one. Weights are distributed downwards so that they multiply
to one at any level. Thus, for Criterion Tree (ii) the global geometric weight for
Income is 0.783 which is got by multiplying 0.894 by the square root of 0.767.
It is apparent from the test that different criteria trees have a considerable effect
on criteria weights. This was true for all nine respondents in this group.
Generally the weights using Criterion Tree (ii) were more out of line than the
other two, as would be expected from the comments. This result confirms what
was discovered in the first test, which is that a poorly structured tree causes
confusion. The next section presents some suggestions on proper structuring.
2.4 Structuring Criteria Trees
A formalisation of some ideas on criteria structuring is now presented using
Saaty’s well known house example (Saaty, 1990b) in which the criteria were size
of house, location to bus lines, neighbourhood, age of house, yard space, modern
facilities, general condition, and financing available. The main idea is that proper
structuring of criteria should take into account any qualitative distinctions which
the decision-maker has identified. The proposed structure is presented in Figure 3.
In this case the financing issue is quite distinct from all the others. Without
finance you cannot get the house. The difference between finance and the other
issues corresponds to the conflict between what is needed to get each house and
the preferences for each one. Some applications of MCDM treat this as a separate
issue to the rest of the problem and use benefit-cost analysis as a second stage
process. This becomes unnecessary with the method described here.
Within the remaining issues, location to bus lines (transportation),
neighbourhood and yard space form a comparable subgroup; which might be
called surroundings. The other subgroup corresponds to more specifically house
issues. Within the house cluster the size of the house is very different to issues to
do with age of house, general condition and modern facilities.
Generally this qualitative clustering and ordering of the criteria is done in a ‘top
down’ manner after the main differentiating attributes between the alternatives
have been discovered. By distinguishing out the criteria which are very different,
such as financing in this case, one can help to clarify the decision maker’s
thinking particularly. This helps later with trade-offs. A frequent comment from
the students was that having to do this forced them to clarify their thoughts.
Presenting the issues on any level on a qualitatively logical order may also help.
Thus financing comes before new home. Likewise within surroundings one
should begin with the more technical transportation, then neighbourhood which
has more to do with other people and lastly yard space which is more a
description of the situation of the house. The latter two could be interchanged if
yard space meant a play area for children and neighbourhood corresponded to
ambience.
Satisfaction With House Financing New Home House Surroundings Size Of Maintenance Trans- Neighbour- Yard House And Value Portation Hood Space Age Of General Modern House Condition Facilities
Figure 3. Hierarchy for satisfaction with house
2.5 Relevance of Nomology
Nomology (Brugha 1998a, 1998b and 1998c) offers a basis for understanding the
structures which underlie qualitative difference. It has three main dimensions:
Adjusting, Convincing and Committing. The kinds of problems considered here,
course choice and house purchase, involve a combination of convincing oneself
about various issues and committing oneself to making a choice. Committing has
three levels: need, preference and value. Convincing has three levels: technical or
self, other people and situational. These are presented in Figure 4 along with a
revision of Maslow’s Hierarchy of Needs to show the generic nature of this
structure. Proper structuring of a multi criteria problem should take account of the
nomological rules based on which the criteria clusters are formed. In this case it
provides the basis for differentiating the various levels.
Consider the structuring that was done for the house choice. The hierarchy in
Figure 3 shows that the biggest qualitative distinction is at the top of the hierarchy
with the issue of financing determining whether or not one can afford the house or
not. The major trade-off is about commitment and the financing the buyers have
against the new home they would prefer. At the second level there is a trade-off
to be made between the house itself and the situational aspect of the surroundings.
At the third level down there is the issue about the size of house one will have
versus the work one has to do in terms of maintenance and adding value to it.
This is a question of commitment of one’s work to the house.
Convincing oneself about the house’s maintenance and value is a question of
age of house having more of a technical aspect, the general condition likely to
have more impact on the people in it, and the modern facilities determining the
type of situation that the people will be getting. Convincing oneself about the
house’s surroundings is a question of trading transportation which is more
technical, the neighbourhood and the people in it, and the yard space which is
situational.
Levels of Convincing Technical Others Situational Self End-users Business / Environment Somatic Physical Political Economic Have / Need Levels of Psychic Social Cultural Emotional Committing Do / Preference Pneumatic Artistic Religious Mystical Are / Value
Figure 4. Structure of Development Activities
3. Criteria Weighting Criteria arise from the differences between the alternatives. We would propose
that the third phase of the process should start with the comparative evaluation of
the alternatives with respect to the lowest level criteria. Intuitively, it makes sense
to not mix evaluations which are qualitatively quite different and distinct from
each other, for instance, in the case of the house above, to compare general
condition with neighbourhood, or yard space with financing available. We
would propose that the identification of criterion weights within each category and
on each level that is relevant to a choice process should be carried out at the one
time, working from the bottom up, if possible. Then the more macro weighting
process could be done between the synthesised sets. Occasionally this could lead
to only two items being compared at one level. For example, in the house case
there would be only one comparison between house and surroundings, and then
one between financing and new home. This has the benefit of requiring the
decision-maker to answer fewer questions. It might create anxiety that the result
would be over-dependent on single judgements such as on the financing versus
new home question. In our opinion this problem can be dealt with most easily by
using sensitivity analysis. As it was, in the actual application (Saaty 1990b, p.17)
there was some surprise about the emergence of the least desirable house with
respect to financing as most preferred. It is possible that our suggested approach
might have helped in this case.
The benefit of working from the bottom up is that, through working with the
alternatives and the criteria, the decision-maker learns more about what they mean
and so scores them accurately. We would also suggest that it makes more sense to
have decision-makers work firstly with criteria which are qualitatively close and
finish up with those which are very different from each other. These latter are the
more difficult judgements, and the trade-offs most crucial to the decision.
Decision makers should be able to see the synthesised scores of the alternatives
for each criteria cluster as these scores are synthesised. Working interactively can
allow for ‘outranking’ or elimination of alternatives which scored very poorly on
some criterion. It also facilitates sensitivity and benefit cost analyses.
A possible alternative scoring approach is to compare the importance of the
group of surroundings issues with one another and then extend the comparison to
the individual house issues. This might be difficult because of mixing different
qualitative changes from criteria to criteria. It also makes more demands on the
patience of the decision-maker because of the increased number of questions.
A variation on this would be to order the criteria qualitatively within any level
and carry out comparisons between levels using the criteria that are qualitatively
nearest to each other. For instance, higher criteria within an one category could be
compared with lower criteria within a higher category, hopefully producing a
seamless join. Within surroundings transportation would be lower than
neighbourhood or yard space and so closer to some of the house criteria. Within
maintenance and value modern facilities is a higher criterion than age or general
condition. Thus one would expect it would be relatively easy for a decision-maker
to compare the relative importance of the modern facilities and transportation
criteria. In fact the first is about the convenience within the home and the second
is about convenience of travel to and from the home, two qualitatively similar
issues. Likewise size of house is very close to age of house. From Figure 3 one
can see that a ‘qualitative frontier’ has been created. If one was uncomfortable
about developing clusters of criteria one could compare criteria which were
nearby on the frontier.
Structuring criteria by working from the bottom up the tree in clusters, the first
and recommended method, has operational advantages. The number of
comparisons are few because the clusters are usually small. This facilitates
information gathering and reduces inconsistency. The challenge it presents the
decision advisor particularly is to analyse the structure of the decision maker’s
criteria before gathering the data. The increase in the complexity of the analysis
offsets the operational advantages. The benefits from qualitatively structuring the
objectives come more from the validity of the process, how it is truer to the
processes of the decision maker’s mind.
Bottom-up evaluation of alternatives may convert the weighting process into
two phases. It may be necessary to convert quantitative scores on attributes of
alternatives into scores which match some criterion. If one of the houses is very
much bigger than the others it might be appropriate to use a utility function to take
account of diminishing returns from house size. Where the yard space of one of
the houses was very inconvenient or wasteful of space one might convert its space
into terms which were expressed in terms of the yard space of another house. The
many modelling possibilities for scoring alternatives on particular attributes
should not influence how one should synthesise the relative importance of
multiple criteria, that is where the distinctions incorporate a qualitative aspect.
In the second set of student tests the criteria were compared using AHP type
relative measurement, while the attributes of the alternatives with regard to each
lowest level criterion were scored using a SMART type utility scale. If adopted
generally, this approach would have the effect of introducing compatibility
between AHP and multi-attribute utility theory (MAUT), as mentioned by Dyer
(1990).
3.1 Naming Clusters
In the first test (Figure 1) and in the house purchase example (Figure 3) the criteria
clusters were given new names. The group running the second test preferred not
to do this because of the fear that it would introduce new issues into the decision.
It is most important that the respondents understanding of the issues be used when
naming clusters. Respondents generally seem to be more comfortable if clusters
are presented in the language they have been using in the process. The greatest
difficulties arose when comparing clusters whose meanings were not
homogeneous and consequently did not facilitate the trade-offs. For example with
Criterion Tree (ii) in Figure 2, comparing Income / Challenge against Education /
Experience was seen as difficult. On the other hand comparing Income against
Challenge / Education / Experience was seen as easier.
3.2 Experiments With Alternative Forms of Weighting
Some of the tests carried out in University College Dublin involved comparing the
AHP with SMART. AHP’s 1 to 9 scale caused difficulties leading respondents to
re-consider some of their first answers. SMART’s visible scale was liked.
An second experiment was carried out whereby respondents were asked to rank
their criteria from the least preferred to the most preferred. Then they were asked
to fill in an AHP type table with numbers greater than one corresponding to how
much more they preferred one to another, either directly or in percentage terms.
Respondents found this difficult.
This experiment was refined by giving respondents scale bars to look at with the
lower one given a “length” of 100; they were required to fill in the value, over
100, for the one being compared with it. This was welcomed. This author had the
suspicion that it might work well for relative scores in the order of 1 to 1.5 times
the lower value, but might break down for higher values.
The current belief is that scoring the relative importance of criteria from the
highest downwards might more easily deal with large variations in relative scores.
4. Conclusion
The research described here focused on personally relevant multi criteria
decisions. By emphasising respect for the decision processes within the mind of
decision makers some developments have been made and tested with regard to the
procedures used for constructing criteria trees, for synthesising the weights of
criteria, and for choosing preferred alternatives. Some small tests mainly amongst
college students indicate a good response and enthusiasm for the improvements
from both respondents and students who carried out the tests as part of their course
assignments. Criteria trees that are structured to reflect the various small and large
qualitative distinctions in the minds of the decision makers help decision makers
to clarify their thoughts. They also make scoring trade-offs easier and reduce
inconsistency. The use of visible bands or bars to help score trade-offs was also
seen as an improvement on using numbers alone, including the AHP 1 to 9 scale.
The suggestions proposed here can be summarised as follows. Criteria and
alternatives should be seen as intertwined. Differences between the alternatives
generate the relevant criteria. Consequently including or excluding an alternative
may have a consequence for the criteria. The initial list of criteria generated by
the alternatives should be structured on a ‘top down’ basis from the most
qualitatively distinct to the least. Alternatives should be scored on the lower level
criteria. Then trade-off scores should be calculated on the criteria moving ‘bottom
up’ towards the overall goal. Decision makers should be made aware of the entire
process as it happens. In this way they can revise their judgements where
necessary. Also, occasionally it may be possible to ‘outrank’ or eliminate an
alternative which scores poorly on some clustered criterion. At the end of the
process, if the major issue is a trade-off of benefits against costs, the procedure
takes on the appearance of a benefit-cost analysis.
It is believed that these conclusions apply where the multiplicity of criteria in a
decision is based on qualitative difference, particularly to models such as the AHP
whose foundations are based in psychophysics. The research also indicates the
relevance of Nomology for guiding the qualitative structuring of a criteria tree.
References Barzilai, J.; Cook, W. D. and Golany, B. (1987) Consistent Weights for
Judgements Matrices of the Relative Importance of Alternatives, Operations
Research Letters, 6 (3), 131-134.
Barzilai, J.; Cook, W. D. and Golany, B. (1992) “The Analytic Hierarchy Process:
Structure of the Problem and its Solutions” in Systems and Management Science
by Extremal Methods, Phillips, F. Y. and Rousseau, J. J. (eds.), Kluwer
Academic Publishers, 361-371.
Barzilai, J. and Golany, B. (1994), AHP Rank Reversal, Normalisation and
Aggregation Rules, INFOR, 32 (2), 5 -64.
Belton, V. and Gear, A. E. (1983), On the Shortcoming of Saaty’s method of
Analytic Hierarchies, Omega, 11, 228-230.
Belton, V. and Gear, A. E. (1985) The legitimacy of Rank Reversal - A comment,
Omega, 13, 143-144.
Brugha, C. (1998a), The structure of qualitative decision making, European
Journal of Operational Research, 104 (1), pp 46-62.
Brugha, C. (1998b), The structure of adjustment decision making, European
Journal of Operational Research, 104 (1), pp 63-76.
Brugha, C. (1998c), The structure of development decision making, European
Journal of Operational Research, 104 (1), pp 77-92.
Crawford, G. and Williams, C. (1985) A Note on the Analysis of Subjective
Judgement Matrices, Journal of Mathematical Psychology, 29, 387-405
Dyer, J.S. (1990), Remarks on the Analytic Hierarchy Process, Management
Science, 36 (March), 249-258.
Gescheider, G.A. (1985) Psychophysics Method, Theory, and Application,
Lawrence Erlbaum Associates, Publishers, New Jersey.
Hamilton, W. (1877), Lectures on Metaphysics, Vols. 1 and 2, 6th Ed., in
Lectures on Metaphysics and Logic, London: William Blackwood and Sons.
Holder, R. D. (1990), Some comments on the Analytical Hierarchy Process, J.
Opl. Res. Soc. 41 (11), 1073-1076.
Lootsma, F. A. (1993), Scale sensitivity in the Multiplicative AHP and SMART,
Journal of Multi-Criteria Decision Analysis, 2, 87-110.
Lootsma, F.A. (1996), “A model of the relative importance of the criteria in the
Multiplicative AHP and SMART”, European Journal of Operational Research,
94, 467-476.
Saaty, T.L. (1980), The Analytic Hierarchy Process, McGraw-Hill, New York.
Saaty, T.L. (1990a), Multicriteria Decision-Making: the Analytic Hierarchy
Process, The Analytic Hierarchy Process Series Vol. 1, RWS Publications.
Saaty, T.L. (1990b), How to make a decision: the Analytic Hierarchy Process,
European Journal of Operational Research, 48, 9-26.
Saaty, T.L. (1994), Highlights and critical points in the theory and application of
the Analytic Hierarchy Process, European Journal of Operational Research, 74,
426-447.
Saaty, T.L. (1996), "Ratio Scales are Fundamental in Decision Making", ISAHP
1996 Proceedings, Vancouver, Canada, July 12-15, 146-156.
Schenkerman, Stan (1994), Avoiding rank reversal in AHP decision-support
models, European Journal of Operational Research, 74, 407-419.
Schoner, B. and Wedley, W.C. (1989), Ambiguous criteria weights in AHP:
consequences and solutions, Decision Sciences, 20, 462-475.