European Journal of Operational Research 232 (2014) 607–612

Contents lists available at ScienceDirect

European Journal of Operational Research

journal homepage: www.elsevier .com/locate /e jor

Decision Support

Preference inference with general additive value models and holisticpair-wise statements

0377-2217/$ - see front matter � 2013 Elsevier B.V. All rights reserved.http://dx.doi.org/10.1016/j.ejor.2013.07.036

⇑ Corresponding author. Address: Econometric Institute, Erasmus UniversityRotterdam, PO Box 1738, The Netherlands. Tel.: +31 10 408 1260.

E-mail addresses: spliet@ese.eur.nl (R. Spliet), tervonen@ese.eur.nl (T. Tervo-nen).

Remy Spliet, Tommi Tervonen ⇑Econometric Institute, Erasmus School of Economics, Erasmus University Rotterdam, The Netherlands

a r t i c l e i n f o

Article history:Received 14 January 2013Accepted 23 July 2013Available online 30 July 2013

Keywords:Multiple criteria analysisDecision analysisPreference learningMulti-attribute value theoryRobust ordinal regression

a b s t r a c t

Additive multi-attribute value models and additive utility models with discrete outcome sets are widelyapplied in both descriptive and normative decision analysis. Their non-parametric application allowspreference inference by analyzing sets of general additive value functions compatible with the observedor elicited holistic pair-wise preference statements. In this paper, we provide necessary and sufficientconditions for the preference inference based on a single preference statement, and sufficient conditionsfor the inference based on multiple preference statements. In our computational experiments all infer-ences could be made with these conditions. Moreover, our analysis suggests that the non-parametricanalyses of general additive value models are unlikely to be useful by themselves for decision supportin contexts where the decision maker preferences are elicited in the form of holistic pair-wise statements.

� 2013 Elsevier B.V. All rights reserved.

1. Introduction

We consider a decision problem of (partially) ordering a set ofalternatives that are deterministically evaluated in terms of n > 1attributes. Preferences of the Decision Maker (DM) are assumedto be representable by an additive multi-attribute value functionthat is indirectly defined through holistic pair-wise judgments(i.e. alternative a is weakly preferred over alternative b, a%b). Notethat although we consider, for compatibility with the existing liter-ature, only multi-attribute value models, the results also apply di-rectly to multi-attribute utility models with discrete outcome sets.The Robust Ordinal Regression (ROR) methodology (Corrente,Greco, Kadzinski, & Słowinski, in press; Corrente, Greco, &Słowinski, 2013; Greco, Mousseau, & Słowinski, 2008; Greco,Kadzinski, Mousseau, & Słowinski, 2011, 2012; Greco, Kadzinski,& Słowinski, 2011; Kadzinski, Greco, & Słowinski, 2012a, Kadzinski,Greco, & Słowinski, 2012b, 2012c, 2013a) enables non-parametricanalyses of sets of preference models compatible with the givenholistic pair-wise judgments. ROR methods supply the DM withtwo kinds of results: possible and necessary preference relationsthat express whether an alternative is weakly preferred over an-other one with some or all compatible preference models, respec-tively. ROR has been implemented initially for the non-parametricanalyses of additive value models in UTAGMS (Greco et al., 2008).

The necessary and possible relations are computed in UTAGMS

by solving Linear Programs (LPs). However, their computation timecan be too high for practical purposes, especially in decision con-texts where the problem needs to be solved repeatedly, or with lar-ger problem sizes. Also, it has not been known what exactly can beinferred through non-parametric analyses of additive models whenthe DMs express preferences as holistic pair-wise statements.Answering this question is relevant for the economical sciencesas a whole, because many regression models implicitly assumean axiomatic foundation in terms of value theory (also known asutility theory with riskless decisions), and as the LP based approachseems to be appropriate also for descriptive decision analysis (Graf,Vetschera, & Zhang, 2013).

In this paper, we prove necessary and sufficient conditions forsingle preference statement inference, and sufficient conditionsfor multiple statement inference (Section 2). We report results ofour computational experiments that measured the amount of dif-ferent types of preference inferences as well as inferences thatcould not be made using our propositions (Section 3). The paperends with a discussion of the propositions and the results.

2. Analysis of the general additive value model

We consider a multi-attribute decision problem where a finiteset of alternatives M is evaluated on a set of attributes indexedwith S = {1, . . . , n}. We denote the evaluation of alternative A 2Mon attribute i with Ai. Without loss of generality, we assume theevaluations to be cardinal and the alternatives Pareto-optimal.

608 R. Spliet, T. Tervonen / European Journal of Operational Research 232 (2014) 607–612

The DM preferences over M are representable with a value functionu : M ! R,

uðAÞP uðBÞ () A%B: ð1Þ

We assume mutual preferential independence of the DM’s prefer-ences (Keeney & Raiffa, 1976) and therefore u is additive and com-posed of partial value functions ui, that are, without loss ofgenerality, assumed to be monotonically increasing,

uðAÞ ¼Xi2S

uiðAiÞ: ð2Þ

The set of all such value functions is U. Let P be the set of weak pref-erence (%) statements provided by the DM. A value function u iscompatible to P if, 8ðA%BÞ 2 P, u(A) P u(B). The set of all value func-tions compatible to P is denoted by UP #U. P is said to be non-con-flicting if UP – ;. In what follows, we assume P to be non-conflicting.

Definition 1. For A, B 2M such that ðA%BÞ R P, ifuðAÞP uðBÞ;8u 2 UP , we say we are able to infer A%B using P.

Note that if uðAÞP uðBÞ;8u 2 UP , then in the UTAGMS terminol-ogy A is necessarily preferred to B.

The conditions we derive for preference inference are based onexamining the partial value function domains whose correspond-ing ranges are constrained in size by the preference statements:

Definition 2. Pþi and P�i are, "i 2 S,

P�i ¼[

ðX%YÞ 2 PYi > Xi

½Xi;Yi� ð3Þ

Pþi ¼[

ðX%YÞ 2 PXi > Yi

½Yi;Xi� ð4Þ

Note that when jPj ¼ 1; Pþi ¼ ½Yi;Xi� if Yi < Xi and Pþi ¼ ; other-wise, and P�i ¼ ½Xi;Yi� if Xi < Yi and P�i ¼ ; otherwise. Using P�iand Pþi , the following lemma provides a condition under whichpreference inference is impossible.

Lemma 1. For A, B 2M, if $k 2 S : Bk > Ak and ½Ak;Bk�� P�k , then9u 2 UP : uðBÞ > uðAÞ.

Fig. 1. An example problem with two preference statements, P ¼ fðX%YÞ; ðV%WÞg,in which the condition of Lemma 1 holds. As the interval [Ak,Bk] is not completelywithin P�k , the value interval [uk(min{D⁄}), uk(max{D⁄})] can be increased by anarbitrary large amount, and therefore 9u 2 UP : uðBÞ > uðAÞ.

Proof. Consider an arbitrary u 2 UP . We construct a new valuefunction u0 by modifying u so that u0 2 UP and u0(B) > u0(A).

Let usum ¼P

i½uiðmaxX2MfXigÞ � uiðminX2MfXigÞ�. Furthermore,let D ¼ ½Ak;Bk� n P�k and let D⁄ be an arbitrary non-empty convexsubset (an interval) of D, i.e. D⁄ is an interval fully contained in D.Let u0i ¼ ui;8i 2 S n fkg, and define u0kðxÞ for an e > 0 as

u0kðxÞ ¼ukðxÞ if x 6 minfD�g;ukðxÞ þ usum þ e otherwise:

�ð5Þ

Observe that u0 differs from u only by the partial value function u0k.Moreover, for each non-empty convex subset½x; y�# P�k ;u

0kðxÞ � u0kðyÞ ¼ ukðxÞ � ukðyÞ, and for

½y; x�# Pþk ;u0kðxÞ � u0kðyÞP ukðxÞ � ukðyÞ. Therefore, 8ðX%YÞ 2 P,

u0ðXÞ � u0ðYÞ ¼X

i2Snfkgu0iðXiÞ � u0iðYiÞ� �

þ u0kðXkÞ � u0kðYkÞ ð6Þ

¼X

i2Snfkg½uiðXiÞ � uiðYiÞ� þ u0kðXkÞ � u0kðYkÞ ð7Þ

PXi2S

½uiðXiÞ � uiðYiÞ� ð8Þ

P 0) u0 2 UP : ð9Þ

Furthermore,

u0ðBÞ � u0ðAÞ ¼X

i2Snfkgu0iðBiÞ � u0iðAiÞ� �

þ u0kðBkÞ � u0kðAkÞ ð10Þ

¼X

i2Snfkg½uiðBiÞ � uiðAiÞ� þ ukðBkÞ þ usum þ e� ukðAkÞ ð11Þ

¼ usum �Xi2S

½uiðAiÞ � uiðBiÞ� þ e ð12Þ

P e; ð13Þ

and e > 0) u0(B) > u0(A). h

Lemma 1 provides a simple condition that can be used forchecking whether 9u 2 UP : uðBÞ > uðAÞ, or in UTAGMS terminology,whether A is not necessarily preferred to B. This is illustrated inFig. 1.

Corollary 1. For A, B 2M, if uðAÞP uðBÞ;8u 2 UP , then½Ai;Bi�# P�i ;8i 2 S : Ai < Bi.

Proof. Assume that uðAÞP uðBÞ;8u 2 UP , and that $k 2 S : Ak < Bk

and ½Ak;Bk�� P�k . Now Lemma 1 implies that9u 2 UP : uðBÞ > uðAÞ, which contradicts the assumption. h

Next we provide another condition under which preferenceinference is impossible. It considers a single preference statementonly.

Lemma 2. For A, B, X, Y 2M and P ¼ fðX%YÞg, if $k 2 S : Ak P Bk andPþk � ½Bk;Ak�, then 9u 2 UP : uðBÞ > uðAÞ.

Proof. If 9t 2 S : ½At ;Bt �� P�t , then Bt > At, and by Lemma 19u 2 UP : uðBÞ > uðAÞ. In the remainder of the proof we considerthe other case in which ½Ai;Bi�# P�i ;8i 2 S. In particular, becauseall alternatives in M are Pareto optimal, $‘ 2 S : B‘ > A‘ and½A‘; B‘�# P�‘ , i.e. Y‘ P B‘ > A‘ P X‘. To complete the proof, we con-struct a value function u and show that u(X) P u(Y) and u(B) > u(A).

For all i 2 Sn{k,‘}, let ui(x) = 0. Furthermore, for an e > 0, define u‘as

u‘ðxÞ ¼e; if x > A‘;

0; otherwise:

�ð14Þ

If Xk > Ak > Yk, define uk as

ukðxÞ ¼2e; if x > Ak;

0; otherwise:

�ð15Þ

R. Spliet, T. Tervonen / European Journal of Operational Research 232 (2014) 607–612 609

Otherwise Xk 6 Ak or Ak 6 YK, and we define uk as

ukðxÞ ¼2e; if x > Yk;

0; otherwise:

�ð16Þ

Note that Xk > Yk because Pþk � ½Bk;Ak�, and therefore uk(Xk) = 2e,uk(Yk) = 0. Moreover, uk(Ak) = uk(Bk). Now,

uðXÞ ¼ u‘ðX‘Þ þ ukðXkÞ ¼ 2e > e ¼ u‘ðY ‘Þ þ ukðYkÞ ¼ uðYÞ; ð17Þ

and therefore u 2 UP . Furthermore,

uðAÞ ¼ u‘ðA‘Þ þ ukðAkÞ ¼2e; if Ak P Xk;

0; otherwise;

�ð18Þ

and

uðBÞ ¼ u‘ðB‘Þ þ ukðBkÞ ¼3e; if Ak P Xk;

e; otherwise;

�ð19Þ

and by combining (18) and (19), it follows that u(B) > u(A). h

Note that Lemma 2 is specific to jPj = 1. In particular, if jPj > 1,preference inference might still be possible even if the lemmaholds for some ðX%YÞ 2 P.

Corollary 2. For A, B 2M and P such that jPj = 1, ifuðAÞP uðBÞ;8u 2 UP, then Pþi # ½Bi;Ai�;8i 2 S : Ai P Bi.

Proof. Assume that uðAÞP uðBÞ;8u 2 UP , and $k 2 S : Ak P Bk andPþk � ½Ak;Bk�. Now Lemma 2 implies that 9u 2 UP : uðBÞ > uðAÞ,which contradicts the assumption. h

Next, we provide necessary and sufficient conditions for prefer-ence inference with a single preference statement.

Theorem 1 (single statement inference). For A, B, X, Y 2M andP ¼ fðX%YÞg, it holds that uðAÞP uðBÞ;8u 2 UP, if and only if "i 2 Sit holds that

P�i � ½Ai;Bi�; if Ai < Bi ð20ÞPþi # ½Bi;Ai�; if Ai P Bi ð21Þ

Proof. First assume that uðAÞP uðBÞ;8u 2 UP . Now (20) follows byCorollary 1 and (21) follows by Corollary 2.

To complete the proof, we show that when (20) and (21) hold,uðAÞP uðBÞ;8u 2 UP . We begin by dividing S into three distinctsubsets: S1 ¼ fi 2 SjAi < Big; S2 ¼ i 2 SjBi 6 Ai; P

þi – ;

� �and

S3 ¼ fi 2 SjBi 6 Ai; Pþi ¼ ;g. Note that Xi 6 Ai < Bi 6 Yi," i 2 S1 by

(20), Bi 6 Yi < Xi 6 Ai,"i 2 S2 by (21), and Xi 6 Yi,"i 2 S3 by (21).

Since uðXÞP uðYÞ;8u 2 UP , andP

i2S3uiðXiÞ 6

Pi2S3

uiðYiÞ, itfollows that

Pi2S1[S2

uiðYiÞ 6P

i2S1[S2uiðXiÞ. Hence,

uðAÞ ¼X

i2S1[S2

uiðAiÞ þXi2S3

uiðAiÞ ð22Þ

PX

i2S1[S2

uiðXiÞ þXi2S3

uiðBiÞ ð23Þ

PX

i2S1[S2

uiðYiÞ þXi2S3

uiðBiÞ ð24Þ

PX

i2S1[S2

uiðBiÞ þXi2S3

uiðBiÞ ð25Þ

¼ uðBÞ; ð26Þ

and therefore uðAÞP uðBÞ;8u 2 UP . h

Example 1. Consider a problem with M = {A,B,X,Y} with the fol-lowing attribute values

Alt

Attr. 1 Attr. 2 Attr. 3 Attr. 4

A

4 2 2 4 B 1 3 1 3 X 3 1 3 1 Y 2 4 4 2

and P ¼ fðX%YÞg. Now Theorem 1 allows us to infer A%B, because½B1;A1� � Pþ1 ; ½A2;B2�# P�2 , and for i 2 {3,4}, Ai P Bi and Pþi ¼ ;. Thisis illustrated in Fig. 2.

The following corollary provides a condition under which nopreference inference is possible.

Corollary 3. For A, B, X, Y 2M and P ¼ fðX%YÞg, if A, B, X and Y arenot all distinct, 9u 2 UP : uðAÞ > uðBÞ and 9u0 2 UP : u0ðAÞ < u0ðBÞ.

Proof. Suppose Y = B, and hence P ¼ fðX%BÞg. First, consider theconditions for uðBÞP uðAÞ;8u 2 UP . Now, (20) cannot hold for ani 2 S : Bi < Ai) Ai 6 Bi, "i 2 S. Moreover, (21) can only hold if Xi = -Bi, "i 2 S : Ai 6 Bi. Because of the Pareto-optimality of alternativesin M, (20) and (21) cannot hold, and therefore9u 2 UP : uðAÞ > uðBÞ.

Second, consider the conditions for uðAÞP uðBÞ;8u 2 UP . Now,(20) simplifies to: Xi 6 Ai if Ai < Bi, and (21) simplifies to: Xi 6 Ai ifBi 6 Ai. Hence, uðAÞP uðBÞ;8u 2 UP () Xi 6 Ai;8i 2 S. As all alter-natives in M are Pareto optimal, this is impossible, and therefore9u0 2 UP : u0ðAÞ < u0ðBÞ.

The proofs for the other cases in which A, B, X and Y are not alldistinct are similar to the case Y = B given above. h

In the next theorem, we provide sufficient conditions for prefer-ence inference using multiple preference statements. They aremore complex than those in Theorem 1. However, both theoremssimilarly apply the given preference statements for bounding attri-bute value differences.

Theorem 2 (sufficient conditions for multiple statement infer-ence). For A;B 2 M; uðAÞP uðBÞ;8u 2 UP, if $P0 # P such that"i 2 S the following hold:

P0þi ¼ ; or P0�i ¼ ; ð27ÞðYi;XiÞ \ ðWi;ViÞ ¼ ;; 8ðX%YÞ; ðV%WÞ 2 P0 ð28ÞP0�i is convex ð29Þif Ai P Bi; then P0þi # ½Bi;Ai� ð30Þif Bi > Ai; then ½Ai;Bi�# P0�i ð31Þ

We would like to emphasize that the condition (28) is definedusing the open intervals (Yi,Xi) and (Wi,Vi) instead of the closedintervals [Yi,Xi] and [Wi,Vi]. The condition states that the intervalsforming P0þi in (3) should be disjoint, with the exception of possibleoverlaps at the endpoints.

Proof. Consider an arbitrary value function u 2 UP . LetSþ ¼ i 2 S P0þi – ;

��� �and S� ¼ i 2 S P0�i – ;

��� �. First, observe that

due to (27), 8ðX%YÞ 2 P0 it holds that

min P0þi� �

6 Yi < Xi 6max P0þi� �

; 8i 2 Sþ; ð32Þmin P0�i

� �6 Xi < Yi 6max P0�i

� �; 8i 2 S�; ð33Þ

and due to (28) and (32),Xi2Sþ

ui max P0þi� ��

� ui min P0þi� �� �

PX

ðX%YÞ2P0

Xi2SþðuiðXiÞ � uiðYiÞÞ: ð34Þ

Fig. 2. Attribute values and their bounding ranges for Example 1; A problem with M = {A,B,X,Y}, 4 attributes, and a single preference statement P ¼ fðX%YÞg that allows us toinfer A%B. The size of the interval ½B1;A1� � Pþ1 is bounded from below and the size of ½A2;B2�# P�2 from above by the preference statement. Attributes 3 and 4 do notcontribute to the bounding as Ai P Bi and Pþi ¼ ;; i 2 f3;4g.

610 R. Spliet, T. Tervonen / European Journal of Operational Research 232 (2014) 607–612

Next, note that "i 2 Sn(S+ [ S�), Xi = Yi for every ðX%YÞ 2 P0, and inparticular ui(Xi) = ui(Yi). Therefore, as by definition u(X) P u(Y) forevery ðX%YÞ 2 P0, it follows thatXi2SþðuiðXiÞ � uiðYiÞÞP

Xi2S�ðuiðYiÞ � uiðXiÞÞ; 8ðX%YÞ 2 P0: ð35Þ

Hence, summing (35) over all preference statements in P0 yieldsXðX%YÞ2P0

Xi2SþðuiðXiÞ � uiðYiÞÞP

XðX%YÞ2P0

Xi2S�ðuiðYiÞ � uiðXiÞÞ; ð36Þ

and applying (29) givesXðX%YÞ2P0

Xi2S�ðuiðYiÞ

� uiðXiÞÞ

PXi2S�

ui max P0�i� ��

� ui min P0�i� �� �

:

ð37Þ

Then, by combining 34, 36 and 37,Xi2Sþ

ui max P0þi� ��

� ui min P0þi� �� �

PXi2S�

ui max P0�i� ��

� ui min P0�i� �� �

; ð38Þ

which can be rewritten asXi2Sþ

ui max P0þi� ��

þXi2S�

ui min P0�i� ��

PXi2Sþ

ui min P0þi� ��

þXi2S�

ui max P0�i� ��

: ð39Þ

Now, as min P0�i� �

6 max P0�i� �

(33), it follows for any subset S0� -

# S� (33) thatXi2Sþ

ui max P0þi� ��

þXi2S0�

ui min P0�i� ��

PXi2Sþ

ui min P0þi� ��

þXi2S0�

ui max P0�i� ��

: ð40Þ

We will use (40) to show that u(A) P u(B). To do so, first observethat due to (30), "i 2 S : Ai P Bi either P0þi ¼ ;, or P0þi – ; in whichcase Bi 6 min P0þi

� �6 max P0þi

� �6 Ai. Also, observe that due to

(31), 8i 2 S : Bi > Ai; P0�i – ; and min P0�i

� �6 Ai < Bi 6 max P0�i

� �.

Based on these observations, (27) allows us to conclude thatP0�i – ;;8i 2 S : Bi > Ai, and therefore

Bi 6min P0þi� �

6 max P0þi� �

6 Ai; 8i 2 Sþ: ð41Þ

Furthermore, we introduce bS� ¼ fi 2 S�jAi < Big. Note that bS� # S�.Now, it holds that

min P0�i� �

6 Ai < Bi 6 max P0�i� �

; 8i 2 bS�: ð42Þ

Moreover, Ai P Bi, 8i 2 S n ðSþ [ bS�Þ. Using (40)–(42) we derive

uðAÞ ¼Xi2Sþ

uiðAiÞ þXi2bS�uiðAiÞ þ

Xi2SnðSþ[bS�ÞuiðAiÞ ð43Þ

PXi2Sþ

ui max P0þi� ��

þXi2bS�

ui min P0�i� ��

þX

i2SnðSþ[bS�ÞuiðBiÞ ð44Þ

PXi2Sþ

ui min P0þi� ��

þXi2S�

ui max P0�i� ��

þX

i2SnðSþ[bS�ÞuiðBiÞ ð45Þ

PXi2Sþ

uiðBiÞ þXi2S�

uiðBiÞ þX

i2SnðSþ[bS�ÞuiðBiÞ ð46Þ

¼ uðBÞ; ð47Þ

and therefore uðAÞP uðBÞ;8u 2 UP . h

Remark 1. For jPj = 1, Theorem 2 simplifies to Theorem 1 as in thatcase the conditions (27)–(29) always hold.

Example 2. Consider a problem with M = {A,B,V,W,X,Y}, the fol-lowing attribute values

Alt

Attr. 1 Attr. 2 Attr. 3 Attr. 4

A

6 6 2 2 B 2 1 5 5 V 5 3 1 3 W 4 2 4 6 X 3 5 3 1 Y 2 4 6 4

and P ¼ fðV%WÞ; ðX%YÞg. In this problem we cannot use Theorem 1to infer that A%B by using only one of the preference statementsðV%WÞ or ðX%YÞ. However, by applying Theorem 2 with P0 = P, wecan infer A%B. This is illustrated in Fig. 3.

3. Computational experiments

We performed computational experiments to assess theamount of preference inferences possible using the lemmas andtheorems given in the previous section, and the amount of infer-

Fig. 3. Attribute values and their bounding ranges for Example 2; A problem with M = {A,B,V,W,X,Y}, 4 attributes, and two preference statements P ¼ fðV%WÞ; ðX%YÞg thatallow us to infer A%B.

R. Spliet, T. Tervonen / European Journal of Operational Research 232 (2014) 607–612 611

ences still missing as we did not provide necessary conditions forthe multiple statement case. We analyzed problems with 5 attri-butes, 10, 20, or 50 random Pareto-optimal alternatives and2, 4, . . . , 40 random preference statements. They are generated insuch a way that "A, B 2M, :((A,B) 2 P ^ (B,A) 2 P). In each param-eter configuration we ran the experiment 10 times to assess the ef-fect of different problem structures. Hence, a total of 600 testinstances were used to measure the amounts of:

1. inferences obtained through the transitivity of % (if A%B andB%C, then through transitivity of % we can infer A%C),

2. impossible inferences (i.e. for a pair of alternativesðA;BÞ; 9u 2 UP : uðBÞ > uðAÞ and therefore we cannot infer thatA%B) discovered with Lemma 1,

2 6 10 14 18 22 26 30 34 38

05

1525

Preference statements, 10 alternatives

% in

ferre

d tra

nsiti

ve0

510

1520

Preference statements, 20 alternatives

% in

ferre

d tra

nsiti

ve0.

01.

02.

03.

0

Preference statements, 50 alternatives

% in

ferre

d tra

nsiti

ve

2 6 10 14 18 22 26 30 34 38

2 6 10 14 18 22 26 30 34 38

Fig. 4. Boxplots for the percentages (in %) of transitive inferences and the impossible infn2 � n � jPj, and the maximum amount of impossible inferences is n2 � n � jPjminus the10 (top), 20 (middle) or 50 (bottom) random Pareto-optimal alternatives, and 2, 4, . . . , 4

3. non-transitive inferences discovered with Theorem 1, and4. inferences not discovered in steps 1 and 3, i.e. that are not part

of the transitive closure and not discoverable with Theorem 1.

The last statistic was measured by comparing the inferencesdiscovered by applying Theorem 1 with the results obtained usingthe full UTAGMS LPs. The experiments were implemented in R andtheir code is publicly available at http://github.com/tommite/pubs-code/tree/master/prefinf-ejor.

The sets of random preference statements were non-conflictingin all test instances ðUP – ;Þ. The maximum amount of inferencesin each instance is n2 � n � jPj. Fig. 4 presents the amount of tran-sitive inferences as percentages of the maximums, and the percent-ages of impossible inferences discovered with Lemma 1. The

020

6010

0

Preference statements, 10 alternatives

% re

mai

ning

Lem

ma

10

2040

6080

Preference statements, 20 alternatives

% re

mai

ning

Lem

ma

1

Preference statements, 50 alternatives

% re

mai

ning

Lem

ma

1

2 6 10 14 18 22 26 30 34 38

2 6 10 14 18 22 26 30 34 38

2 6 10 14 18 22 26 30 34 38

020

4060

80

erences discovered with Lemma 1. The maximum amount of transitive inferences isamount of transitive inferences. The results are for test problems with 5 attributes,0 preferences statements (10 runs for each test configuration).

612 R. Spliet, T. Tervonen / European Journal of Operational Research 232 (2014) 607–612

percentages of transitive inferences increase with the amount ofpreference statements until reaching a saturation (with 10 alterna-tives this is approximately at 22 preference statements) and after-wards they decrease. With only a few preference statements thepercentages of impossible inferences discovered with Lemma 1are reasonably high, but they rapidly decrease when the numberof preference statements included increase. Both of these resultswere expected. What is more interesting is that none of the 600test instances contained non-transitive inferences although Exam-ples 1 and 2 show that such instances exist. Also, there were noadditional inferences discovered with the full UTAGMS LPs, i.e. allthe inferences in all test instances were transitive.

4. Discussion

The assumption of mutual preferential independence, which isrequired for the application of additive value models, has tradition-ally been considered to seldom hold in descriptive analyses of real-life decisions. This has led to the development and use of morecomplex models such as the multiplicative and multilinear ones.However, our computational experiments show that there usuallyexists a set of additive functions that can reproduce the preferenceinformation and still be compatible with the assumption of mutualpreferential independence (none of the test instances containedUP ¼ ;). Therefore, in non-parametric analyses of sets of preferencemodels the choice between additive and non-additive value func-tions cannot be made based on holistic pair-wise preference state-ments alone, but additional assumptions that cannot be derivedfrom the observations are required for choosing between the two.

For normative decision analysis (decision support), our compu-tational experiments were representative in size of problemsencountered in practice; it would be unrealistic to assume thatthe DM would be willing to provide more than 40 preference state-ments for ordering 10 or 20 alternatives. As the tests contained noinstances with inferences apart from the transitivity of %, our anal-ysis might completely describe the non-parametric analyses ofadditive value models using holistic pair-wise preference state-ments. As the conditions for inference with these models (Theo-rems 1 and 2) are quite specific, such inferences are unlikely tobe made. Therefore, such models are probably too general to beof use in normative non-parametric multiple criteria decisionanalysis. On one hand, this questions the practical relevance ofthe robust ordinal regression method UTAGMS (Greco et al., 2008)for decision support, but on the other hand our results justify thelater advances for selecting a single representative value function(Greco et al., 2011; Kadzinski et al., 2012a, Kadzinski, Greco, &Słowinski, 2013b), the rank-based analyses (Kadzinski et al.,2012b, Kadzinski, Greco, & Słowinski, 2013a), and the hybridprobabilistic approaches (Kadzinski & Tervonen, 2013a, 2013b).

Our results are limited by three factors: (i) we were not able toprove necessary conditions for multiple-statement inference, (ii)the computational experiments were restricted in size to instancesrepresentative of real-life decision support problems, but not ofrevealed preference studies, and (iii) extrapolation of our results

is not directly applicable to studies of revealed preferences as theyoften contain noise. The first limitation should be addressed in fur-ther research, and if the necessary conditions become available, theneed for more extensive computational experiments stated in (ii)becomes obsolete. To provide a concrete future research problem,we conclude this paper with a conjecture that Theorem 2 containsalso the necessary conditions for multiple statement inference.

Acknowledgements

The computational tests of this research were performed on theDutch National LISA cluster, and supported by the Dutch NationalScience Foundation (NWO).

References

Corrente, S., Greco, S., Kadzinski, M., & Słowinski, R. (in press). Robust ordinalregression in preference learning and ranking. Machine Learning, doi:http://dx.doi.org/10.1007/s10994-013-5365-4.

Corrente, S., Greco, S., & Słowinski, R. (2013). Multiple criteria hierarchy processwith ELECTRE and PROMETHEE. Omega, 41, 820–846. http://dx.doi.org/10.1016/j.omega.2012.10.009.

Graf, C., Vetschera, R., & Zhang, Y. (2013). Parameters of social preference functions:Measurement and external validity. Theory and Decision, 74(3), 357–382. http://dx.doi.org/10.1007/s11238-012-9312-9.

Greco, S., Kadzinski, M., Mousseau, V., & Słowinski, R. (2011). ELECTREGKMS: Robustordinal regression for outranking methods. European Journal of OperationalResearch, 214(1), 118–135. http://dx.doi.org/10.1016/j.ejor.2011.03.045.

Greco, S., Kadzinski, M., Mousseau, V., & Słowinski, R. (2012). Robust ordinalregression for multiple criteria group decision problems: UTAGMS-GROUP andUTADISGMS-GROUP. Decision Support Systems, 52(3), 549–561. http://dx.doi.org/10.1016/j.dss.2011.10.005.

Greco, S., Kadzinski, M., & Słowinski, R. (2011). Selection of a representative valuefunction in robust multiple criteria sorting. Computers & Operations Research,38(11), 1620–1637. http://dx.doi.org/10.1016/j.cor.2011.02.003.

Greco, S., Mousseau, V., & Słowinski, R. (2008). Ordinal regression revisited:multiple criteria ranking using a set of additive value functions. EuropeanJournal of Operational Research, 191(2), 416–436. http://dx.doi.org/10.1016/j.ejor.2007.08.013.

Kadzinski, M., Greco, S., & Słowinski, R. (2012a). Selection of a representative valuefunction in robust multiple criteria ranking and choice. European Journal ofOperational Research, 217(3), 541–553. http://dx.doi.org/10.1016/j.ejor.2011.09.032.

Kadzinski, M., Greco, S., & Słowinski, R. (2012b). Extreme ranking analysis in robustordinal regression. Omega, 40(4), 488–501. http://dx.doi.org/10.1016/j.omega.2011.09.003.

Kadzinski, M., Greco, S., & Słowinski, R. (2012c). Selection of a representative set ofparameters for robust ordinal regression outranking methods. Computers &Operations Research, 39(11), 2500–2519. http://dx.doi.org/10.1016/j.cor.2011.12.023.

Kadzinski, M., Greco, S., & Słowinski, R. (2013a). RUTA: A framework for assessingand selecting additive value functions on the basis of rank related requirements.Omega, 41(4), 735–751. http://dx.doi.org/10.1016/j.omega.2012.10.002.

Kadzinski, M., Greco, S., & Słowinski, R. (2013b). Selection of a representative valuefunction for robust ordinal regression in group decision making. Group Decisionand Negotiation, 22(3), 429–462. http://dx.doi.org/10.1007/s10726-011-9277-z.

Kadzinski, M., & Tervonen, T. (2013a). Stochastic ordinal regression for multiplecriteria sorting problems. Decision Support Systems, 55(1), 55–66. http://dx.doi.org/10.1016/j.dss.2012.12.030.

Kadzinski, M., & Tervonen, T. (2013b). Robust multi-criteria ranking with additivevalue models and holistic pair-wise preference statements. European Journal ofOperational Research, 228(1), 169–180. http://dx.doi.org/10.1016/j.ejor.2013.01.022.

Keeney, R., & Raiffa, H. (1976). Decisions with multiple objectives: preferences andvalue trade-offs. Cambridge: Cambridge University Press.