Robustness Measures in Criteria Importance Estimation Based on Hamiltonian Search Algorithms 017

8/13/2019 Robustness Measures in Criteria Importance Estimation Based on Hamiltonian Search Algorithms 017

1/37

How robust is the elicitation of criteriaweights

through Simos procedure?

N. Tsotsolas, E. Siskos, N. Christodoulakis


2/37

Support the debate through a case study

Discuss the robustness of Simos procedure

The Simos procedure

The notion of robustness analysis in DM process

Research Aims

This research has been co-financed by the European Union

(European Social Fund) and Greek national funds through theOperational Program "Education and Lifelong Learning"

RobustMCDA


3/37


4/37

The stability of a model or/and of a solution should be assessed and

evaluated each time

The analyst shall be able to have a clear picture regarding the reliability

of the produced results

Stability and reliability shall be expressed using measures which are

understandable by the analyst and the decision maker

Based on these measures the decision maker may accept or reject the

proposed decision model

The need for robustness analysis

EURO XXVI, 26th European Conference nOperational Research, Rome, July 2013


5/37


The Linear Programming can be directly related to the geometry and graph

theory. As for the geometry of the relationship lies in the fact that a system of

inequalities (constraints of LP) define a convex hyper-polyhedron (which isusually bounded). A linear system of nvariables can be represented by aconvex polyhedron of n-dimensions

According to this approach, the search of solutions of LP is equivalent to thetransition from one vertex of the hyper-polyhedron to another. In other words,the basic feasible solutions of LP correspond to the vertices of the hyper-polyhedron.

Post-optimal robustness analysis in LP

1

4

2

3

Optimal

Solution

Initial

Solution


6/37


Finding multiple optimal solutions in LP

Ax b

c x

x

t

z*

0

Multiple Optimal Solutions

max z

s.t.

tc x

Ax b

x 0


7/37EURO XXVI, 26th European Conference nOperational Research, Rome, July 2013

Categories of algorithms finding multiple solutions when proceed withpost-optimality analysis:

Analytical algorithms which promise complete search of all basicfeasible solutions of a hyper-polyhedron. Within the first group wefind two sub-categories of algorithms.

Pivoting methods based on Dantzigs Simplex approach. Non-pivoting methods that do not use the Simplex approach but use

elements of the theory of geometry based on the properties ofintersections between hyper-planes and hyper-polyhedron.

Heuristic algorithms which do not intend to find all solutions of abasic hyper-polyhedron but a representative set of these usingvarious approaches.

Finding multiple optimal solutions in LP


8/37EURO XXVI, 26th European Conference n Operational Research, Rome, July 2013

The V (set of nodes of the graph) contains m-dimension vectors whose

elements are integers (indices of Simplex bases) u=(i1, i2, ..., im) for 1ijm,j=1,2,...,m.

The distance between two different nodes u1=(i1, i2, ..., im) and u2=(k1, k2, ..., km)is d m if d is the number of elements in u2which are different from those ofu1. u1and u2are adjacent if the distance between them is d=1

An arc(u1, u2) U if and only if u1and u2are adjacent. The adjacent nodes uiconstitute set N(ui)

Manas Nedoma Algorithm

u1=(i1, i2, ..., im)u2=(k1, k2, ..., km)d=1

Graph (V,U)


9/37EURO XXVI, 26th European Conference nOperational Research, Rome, July 2013

The number of vertices of the hyper-polyhedron of multiple optimal solutions is

often too big and an exhaustive search requires huge computational effort.Heuristic methods offer a very good alternative to the problem of searchingthousands of solutions which may not be of particular interest to the analyst.

More often the analysts may be only interested in the information which will helphim/her to examine the stability of the an optimal solution or the statisticalvariance of other solutions. For example according to Siskos (1984), the stabilityanalysis may be performed by solving a linear programs (LP) having thefollowing form:

A heuristic approach

The coefficients of the new

constraint in augmented

optimal table of Simplex is

the opposite values of the

marginal values of the

optimal solution table

1

max or [min]

s.t.

z*

0

n

j j

j

p x

t

Ax b

c x

x


10/37

Number of multiple-optimal solutions

1 2int int2 2

n nn m n m

r

m m

n

m 5 50 1000

10 132 3.15E+08 5.94E+20

20 462 4.93E+12 1.16E+36

50 2652 7.01E+19 6.58E+711000 1003002 9.12E+49 #NUM!

5000 25015002 2.06E+67 #NUM!



11/37

x25

60

4

3

2

1

54321 x1

(3,4)

(6,5)

(5,1)

(2,2)

Analytic vs Heuristic

Heuristic

max x1: (6.00, 5.00)max x2: (6.00, 5.00)

min x1: A(2.00, 2.00)

min x2: (5.00, 1,00)

M.V.: C(4.75, 3.25)

Heuristic (without double)max x1: (6.00, 5.00)

min x1: A(2.00, 2.00)

min x2: (5.00, 1,00)

M.V.: C(4.33, 2.67)

AnalyticA(2.00, 2.00)

(5.00, 1,00)

(6.00, 5.00)

(3.00, 4.00)

M.V.: C(4.00, 3.00)

C

C

C



12/37

Simos procedure

In a decision aiding context, knowing the preferences of the Decision Maker (DM) and

determining weights of criteria are very hard questions. Several methods can be used to

give an appropriate value to the weights of criteria.

Simos (1990a,b) proposed a technique allowing any DM (not necessarily familiarized with

multicriteria decision aiding) to think about and express the way in which he wishes to

hierarchise the different criteria of a family Fin a given context.

This procedure also aims to communicate to the analyst the information that DM needs in

order to attribute a numerical value to the weights of each criterion of F, when they are

used in multicriteria outranking methods (such us ELECTRE, PROMETHEE, etc)

The main innovation in this approach consists of associating a playing card with each

criterion. The fact that the person being tested has to handle the cards in order to rankingthem, inserting the white ones, allows a rather intuitive understanding of the aim of this

procedure.



13/37

Simos procedure

Collecting the information:

STEP 1: We give to the DM a set of cards: the name of each criterion is written on each

card. Therefore, we have ncards, nbeing the number of criteria of a family F. We also

give a set of white cards with the same size. The number of the latter will depend on the

users needs

STEP 2: We ask the user to rank these cards from the least important to the mostimportant. So, the DM will rank them in ascending order: the first criterion in the ranking is

the least important and the last criterion in the ranking is the most important. If some

criteria have the same importance (i.e., the same weight), he should build a subset of

cards holding them together with a clip. Consequently, we obtain a complete pre-order on

the whole of the ncriteria.

STEP 3: We ask the DM to think about the fact that the importance of two successive

criteria (or two successive subsets of criteria) in the ranking can be more or less close.

The determination of the weights must take into account this smaller or bigger difference in

the importance of successive criteria. So, we ask him/her to introduce white cards between

two successive cards (or subsets). The greater the difference between the mentioned

weights of the criteria (or the subsets), the greater the number of white cards.



14/37

Simos procedure

Converting the ranks into weights by using Simos procedure

Let us consider a family Fwith 12 criteria: F = {a, b, c, d, e, f, g, h, I, j, k, l}:

(Maystre et al., 1994)



15/37

Criticism of Simos procedure

According to Figueira & Roy (2002) the way Simos recommends to process the information

needed a revision for two main reasons:

Simos procedure elicits only one set of weights that satisfies DMs expressed model.

However, other sets of weights could probably fit better DMs opinion about the relative

importance of criteria. Such sets of weights cannot be obtained by the Simos procedure.

It is based on an unrealistic assumption. This occurs by the lack of an essentialinformation. The procedure limits the set of the feasible weights because it determines

automatically the ratio between the weight of the most important criterion and the weight of

the least important one in the ranking (let call z the value of this ratio). Nevertheless, this

ratio reflects no preference of the DMs and furthermore it depends on the number of cards

in the same subset.

It leads to process criteria having the same importance (i.e., the same weight) in a not

robust way. If somebody tries to re-order the cards between two subsets, he/she realise

that the distance (difference of weights) between the subsequent sub-sets are changed in

an uncontrolled way. This fact occurs because the difference of weights between two

successive subsets of criteria is automatically influenced by the existence of the number of

cards in these successive subsets. The user have not a real or absolute perception of the

way in which the numerical values are determined by the procedure.EURO XXVI, 26th European Conference nOperational Research, Rome, July 2013


16/37

Revised Simos procedure

Figueira & Roy (2002) proposed a revision of Simos procedure:

The revised Simos procedure uses the same data collection method.

It introduced a new kind of information asking the DM to state how many times the last

criterion is more important than the first one in the ranking (ratio z).This ration is used in

order to define a fixed interval between the weights the criteria or their sub-sets. Let u

denotes this interval:

where eis the number of different weight classes (meaning, single card, subsets of cards

and white cards). In the following table the non-normalized weights for z= 6.5 are calculated.



17/37


Determining the normalized weights of each criterion for z= 6.5:



18/37

Calculate Simos weights using LP

We proceed with an indirect assessment of the weights using pairwisecomparisons according to the information given by the DM using the cards. In

other words, we try to reproduce the given pre-order of the ncriteria using LP

formulation.

By solving this LP we try to find the weights that satisfy all the constraints of the

pre-order.

Given the fact that there is always the possibility that the formulated LP has no

feasible solutions due to probable inconsistencies of DMs given information, we

use a goal-programming LP approach in order to produce a feasible set of

weights, by minimising all deviations per pairwise comparison.



19/37

Evaluating Robustness of Simos Procedure

Indirect assessment of the weights using pairwise comparisons of the sameexample discussed by Figueira and Roy

Such an LP system creates between the weights pithe following constraints:

p3= p7p7= p12p4> p3w1> p4p2> w1p2= p6

p6

= p9p9= p10

p5> p2p1> p5p1= p8p11> p1

p1+ p2+ p3+ p4+ p5+ p6+ p7+ p8+ p9+ p10+ p11+ p12= 1


Strict inequalities require the introduction of a threshold, let say = 0.01. Because such a

system may not be feasible, we insert 1-2 error variables following the approach of goal

programming and we minimize the sum of these errrors in a specialized LP:

[min] z = s1a + s1b + s2a+ s2b+ s3+ sw1a+ sw1b+ s4a+ s4b+ s5a+ s5b+ s6a+ s6b+ s7

+ s8+ s9a+ s9b+ s10+ s11a+ s11bs.t.

p3- p7- s1a+ s1b= 0

p7- p12- s2a+ s2b= 0

p4- p3 + s3>=0.01

w1- p4+ sw1a>= 0.01

p2- w1+ sw1b>= 0.01

p2- p6 - s4a+ s4b= 0

p6- p

9- s

5a+ s

5b= 0

p9- p10- s6a+ s6b= 0

p5- p2+ s7>=0.01

p1- p5+ s8>=0.01

p1- p8- s9a+ s9b= 0

p11- p1+ s10>=0.01

p1+ p2+ p3+ p4+ p5+ p6+ p7+ p8+ p9+ p10+ p11+ p12- s11a+ s11b= 1


20/37


p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12

Simos 13.00 9.00 2.00 5.00 12.00 9.00 2.00 13.00 9.00 9.00 15.00 2.00

Manas- Nedoma 14.84 7.25 0.80 2.86 10.65 7.25 0.80 14.84 7.25 7.25 25.41 0.80

Max - Min 15.20 6.92 0.93 3.17 11.41 6.92 0.93 15.20 6.92 6.92 24.57 0.93


The set of the weights of the Figueira & Roys example using:

Simos procedure

LP formulation (with multiple optimal solutions because z= 0 (sum of goal

programming errors)

With Manas-Nedoma analytical algorithm

With Max-Min heuristic algorithm


21/37

27.33

11.38 5.58 8.44

20.7511.38

5.58

27.33

11.37 11.37

73.00

5.58

14.84

7.250.80 2.86

10.65

7.25 0.80

14.84

7.25 7.25

25.41

0.805.00 3.00

0.00 1.00

4.00

3.00 0.005.00 3.00 3.00

11.58

0.000

10

20

30

40

50

60

70

80

90

100

p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12

Max Average Min

27.33

11.385.58 8.44

20.7511.38

5.58

27.33

11.38 11.38

73.00

5.5815.20

6.920.93

3.1711.41

6.920.93

15.206.92 6.92

24.57

0.935.003.00 0.00

1.004.00

3.000.00

5.00 3.00 3.00

11.58

0.000

10

20

30

40

50

60

70

80

90

100

p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12

Max Average Min


MAX-MIN

vs

Manas-Nedoma


Number of solutions - vertices: 147ASI: 0.809

(ASI: Average Stability Index which represents the mean value

of the normalised standard deviation of the estimated weights

Number of solutions: 24

ASI: 0.811


22/37

Evaluating Robustness of Revised Simos Procedure

LP formulation according to Revised Simos procedure:

p3= p7p7= p12

p4> p3w1> p4p2> w1p2= p6

p6= p9p9= p10

p5> p2p1> p5p1= p8p11> p1p1+ p2+ p3+ p4+ p5+ p6+ p7+ p8+ p9+ p10+ p11+ p12= 1p11= 6.5 p3


LP formulation according to Revised Simos procedure using goal programming approach:

[min] z = s1a + s1b + s2a+ s2b+ s3+ sw1a+ sw1b+ s4a+ s4b+ s5a+ s5b+ s6a+ s6b+ s7

+ s8+ s9a+ s9b+ s10+ s11a+ s11b + s12a+ s12b

s.t.

p3- p7- s1a+ s1b= 0

p7- p12- s2a+ s2b= 0

p4- p3 + s3>=0.01w1- p4+ sw1a>= 0.01

p2- w1+ sw1b>= 0.01

p2- p6 - s4a+ s4b= 0

p6- p9- s5a+ s5b= 0

p9- p10- s6a+ s6b= 0

p5- p2+ s7>=0.01p1- p5+ s8>=0.01

p1- p8- s9a+ s9b= 0

p11- p1+ s10>=0.01

p1+ p2+ p3+ p4+ p5+ p6+ p7+ p8+ p9+ p10+ p11+ p12- s11a+ s11b= 1p11- 6.5 p3- s12a+ s12b=0


23/37


p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12

Revised Simos 13.20 8.80 2.40 4.50 11.00 8.80 2.40 13.20 8.80 8.80 15.30 2.40

Manas- Nedoma 13.44 8.23 2.64 4.45 10.69 8.23 2.64 13.44 8.23 8.23 17.16 2.64

Max - Min 13.04 7.92 2.81 4.82 10.74 7.92 2.81 13.04 7.92 7.92 18.27 2.81

z = p11/p3

Revised Simos 6.375 (!!)

Manas- Nedoma 6.50

Max - Min 6.50



24/37

18.39

10.35

4.17

7.79

15.40

10.35

4.17

18.39

10.35 10.35

27.11

4.17

13.04

7.92

2.81

4.82

10.74

7.92

2.81

13.04

7.92 7.92

18.27

2.81

9.17

5.68

1.97

3.05

6.98

5.68

1.97

9.17

5.68 5.68

12.79

1.970

5

10

15

20

25

30

p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12

Max Average Min

18.39

10.35

4.177.79

15.40

10.35

4.17

18.39

10.35 10.35

27.11

4.17

13.44

8.23

2.64 4.45

10.69

8.23

2.64

13.44

8.23 8.23

17.16

2.64

9.17

5.68

1.97 3.05

6.98

5.68

1.97

9.17

5.68 5.68

12.79

1.970

5

10

15

20

25

30

p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12

Max Average Min


MAX-MIN

vs

Manas-Nedoma


Number of solutions - vertices: 137ASI: 0.922


ASI: 0.920


25/37

A Case Study

Extensions of metro lines in Athens

ATTIKO METRO S.A. the company that manages the Athens Metro network wants to rationally

prioritize extension projects of the existing network of the Athens metro that are in the

implementation phase. They agreed on the following 6 criteria for ranking the 4 alternative

actions:

Social Criteria

g1: The number of residents and employees that would be served per km of extension line.

g2: Number of passengers per km of extension line per day.

Financial Criteria

g3: Construction cost per km of extension line (in million ).

g4: ROI Index (%).

Technical and organizational criteria

g5: Coherence Index of the network (scores given of experts from 1 to 10).

g6: Index of urban regeneration (scores given of experts from 1 to 10).EURO XXVI, 26th European Conference nOperational Research, Rome, July 2013


26/37

A Case Study

Multicriteria evaluation table


Extension g1 g2 g3 g4 g5 g6

300.000 40.000 -50 10 8 5

180.000 35.000 -35 15 5 8

C 100.000 20.000 -25 12 5 6

D 150.000 30.000 -30 15 5 6


27/37

A Case Study (Simos Procedure)

LP formulation according to Simos procedure:

p5 - p6= 0

p4 - p5>= 0.01

p1- p4>= 0.01p1- p2= 0

p3- p1>= 0.01

p1 + p2+ p3+ p4+ p5+ p6= 1


LP formulation according to Simos procedure using goal programming approach:

[min] z = s1a+s1b+s2+s3+s4a+s4b+s5+s6a+s6bs.t.

p5- p6-s1a+s1b= 0 ;

p4- p5+s2> = 0.01 ;

p1- p4+s3> = 0.01 ;p1- p2s4a+s4b= 0 ;

p3- p1+s5>= 0.01 ;

p1+ p2+ p3+ p4+ p5+ p6s6a+s6b= 1

p1 p2 p3 p4 p5 p6

Simos 21.00 21.00 29.00 15.00 7.00 7.00

Manas- Nedoma 19.52 19.52 43.05 11.01 3.44 3.44

Max - Min 19.25 19.25 43.25 10.58 3.83 3.83


28/37


Manas-Nedoma vs MAX-MIN


Number of solutions - vertices: 48

ASI: 0.662


ASI: 0.659

p1 p2 p3 p4 p5 p6

2.00 2.00 95.00 1.00 0.00 0.00

17.33 17.33 18.33 16.33 15.33 15.33

25.00 25.00 26.00 24.00 0.00 0.00

32.67 32.67 33.67 1.00 0.00 0.00

Different set of weights

(post-optimal solutions)

(%):

32.67 32.67

95.00

24.0015.33 15.33

19.57 19.57

43.57

10.903.19 3.19

2.00 2.00

18.33

1.00 0.00 0.000

10

20

30

40

50

60

70

80

90

100

p1 p2 p3 p4 p5 p6

Max Average Min

32.67 32.67

95.00

24.0015.33 15.33

22.70 22.70

32.90

12.50

4.60 4.602.00 2.00

18.33

1.00 0.00 0.000

10

20

30

40

50

60

70

80

90

100

p1 p2 p3 p4 p5 p6

Max Average Min


29/37



Rank SimosManas

NedomaMax-Min 1 2 3 4

1st , D , D C

2nd ,D D , D , D

3rd , C , C D

4th C C C C

Ranking of 4 alternatives using PROMETHEE method


30/37

A Case Study (Simos Procedure with white cards)

LP formulation according to Simos procedure:

p5- p6= 0

w1- p5>= 0.01

p4- w1>= 0.01p1- p4>= 0.01

p1 - p2= 0

w2 - p1>= 0.01

p3- w2>= 0.01

p1+ p2+ p3+ p4+ p5+ p6= 1


LP formulation according to Simos procedure using goal programming approach:

[min] z = s1a-s1b- sw1a- sw1b-s3-s4a-s4b-sw2a- sw2b-s6a-s6bs.t.

p5- p6-s1a+s1b= 0

w1- p5+ sw1a>= 0.01

p4- w1+ sw1b>= 0.01p1- p4+s3> = 0.01

p1- p2s4a+s4b= 0

w2- p1+sw2a>= 0.01

p3- w2+sw2b>= 0.01

p1+ p2+ p3+ p4+ p5+ p6s6a+s6b= 1

p1 p2 p3 p4 p5 p6

Simos 20.00 20.00 33.00 15.00 6.00 6.00

Manas- Nedoma 17.76 17.76 48.41 11.98 2.05 2.05

Max - Min 19.31 19.31 43.06 11.06 3.63 3.63


31/37





ASI: 0.672


ASI: 0.677

p1 p2 p3 p4 p5 p6

3.00 3.00 92.00 2.00 0.00 0.00

17.50 17.50 19.50 16.50 14.50 14.50

24.75 24.75 26.75 23.75 0.00 0.00

32.00 32.00 34.00 2.00 0.00 0.00

32.00 32.00

92.00

23.7514.50 14.50

22.58 22.58

33.28

12.88

4.35 4.353.00 3.00

19.50

2.00 0.00 0.000

10

20

30

40

50

60

70

80

90

100

p1 p2 p3 p4 p5 p6

Max Average Min

32.00 32.00

92.00

23.7514.50 14.50

17.76 17.76

48.41

11.982.05 2.053.00 3.00

19.50

2.00 0.00 0.000

10

20

30

40

50

60

70

80

90100

p1 p2 p3 p4 p5 p6

Max Average Min



(%):


32/37



Rank SimosManas

NedomaMax-Min 1 2 3 4

1st D , D C

2nd , D , C D , D , D

3rd , C D

4th C C C C



33/37

A Case Study (Revised Simos Procedure)

LP formulation according to Revised Simos procedure:

p5- p6= 0

w1- p5>= 0.01p4- w1>= 0.01

p1- p4>= 0.01

p1 - p2= 0

w2 - p1>= 0.01

p3- w2>= 0.01

p1+ p2+ p3+ p4+ p5+ p6= 1


LP formulation according to Revised Simos procedure using goal programming

approach:

[min] z = s1a-s1b- sw1a- sw1b-s3-s4a-s4b-sw2a- sw2b-s6a-s6bs.t.

p5- p6-s1a+s1b= 0

w1- p5+ sw1a>= 0.01p4- w1+ sw1b>= 0.01

p1- p4+s3> = 0.01

p1- p2s4a+s4b= 0

w2- p1+sw2a>= 0.01

p3- w2+sw2b>= 0.01

p1+ p2+ p3+ p4+ p5+ p6s6a+s6b= 1

p1 p2 p3 p4 p5 p6 z=p3/p5

Revised Simos 19.80 19.80 34.00 15.00 5.70 5.70 5.96(!!)

Manas- Nedoma 19.11 19.11 35.73 14.14 5.95 5.95 6.00

Max - Min 19.08 19.08 36.43 13.26 6.07 6.07 6.00


34/37





ASI: 0.844


ASI: 0.842

p1 p2 p3 p4 p5 p6

11.36 11.36 50.18 10.36 8.36 8.36

22.69 22.69 24.69 21.69 4.12 4.12

27.14 27.14 29.14 6.86 4.86 4.86

27.14 27.14

50.18

21.69 8.36 8.3619.08 19.08

36.43

13.266.07 6.0711.36 11.36

24.69

6.864.12 4.120

10

20

30

40

50

60

70

80

90

100

p1 p2 p3 p4 p5 p6

Max Average Min

11.36 11.36

24.69

5.004.12 4.12

27.14 27.14

50.18

21.698.36 8.36

19.28 19.28

35.35

14.315.89 5.89

0

10

20

30

40

50

60

70

80

90100

p1 p2 p3 p4 p5 p6

Min Max Average



(%):


35/37



Rank SimosManas

NedomaMax-Min 1 2 3

1st

C, D

2nd , D , D , D , D , D

3rd

4th C C C C C



36/37

Conclusions

1

The results produced by

using Simos procedure

cannot be considered as

robust. Several sets ofrealistic weights are

excluded and the DM is

not informed for this issue.

2

The additional information

provided by the DM under


increase the robustness.Nevertheless, the DM is

not informed again for the

existence of several other

set of weights which also

respect the DMs

preferences.

3

There is a need to

provide crucial

information to DM in

relation to the stability ofthe set of weights

proposed for his/her

model:

ASI

+

Weight Variation withVisualization



37/37

Documents

Robustness Measures in Criteria Importance Estimation Based on Hamiltonian Search Algorithms 017