yaityondo

bleojechor pcisiesrmitartpod o

ring sevnizatioith thisojects t. An eortfolied) tod), wouge. Forore th

Decision Support Systems 54 (2012) 534550

Contents lists available at SciVerse ScienceDirect

Decision Supp

.eprioritize the projects in decreasing order of benet-to-cost ratios(assuming that there are no interactions and hence it is meaningful toassess costs and benets for each project) and proceed down the listuntil the available budget were exhausted [7,16,22,34,36,45,56,62].The portfolio selected by this approachwould produce the highest ben-et for the money spent, but would not necessarily deliver the maxi-mum benet for the money available. Alternatively, the managercould pursue an optimization approach, in which the portfolio with

Chapter 8.1] and Kleinmuntz [37] briey discuss pros and cons ofthese two approaches. Complementary arguments favoring each oneof them are presented in Section 2.

Resource allocation decisions often require managers to considermultiple quantitative and qualitative benet dimensions (or criteria).In a previous paper [46], we studied commercial off-the-shelf softwarefor multicriteria portfolio analysis that aggregates multiple benetcriteria additively: Equity [17], HiPriority [40], Logical Decisions Port-the highest benet for the budget available wa (knapsack) mathematical programming promizes cumulative benet without exceedin[2831,34,37,53,64].

Corresponding author. Tel.: +351 214233523; fax:E-mail addresses: joao.lourenco@ist.utl.pt (J.C. Loure

(A. Morton), carlosbana@ist.utl.pt (C.A. Bana e Costa).

0167-9236/$ see front matter 2012 Elsevier B.V. Allhttp://dx.doi.org/10.1016/j.dss.2012.08.001an one million portfolios

ion strategy would be to

the optimization approach. The aforementioned exclusion makes theformer approach appear to be an intuitive, approximate approachto solving the optimization problem [18, p. 260]. Kirkwood [34,(precisely, 220=1,048,576).A less strenuous and more practical selectSuppose that a manager is consideeach expected to add value to his orgafunds to capitalize all of them. Faced wwould like to select the portfolio of przation with the best value for moneypossible portfolios, from the empty pare funded and no benets are realizwould require all projects to be fundethe number of projects were not too larset of just 20 projects could result in meral indivisible projects,n, but does not have theconstraint, themanagerhat provides the organi-xhaustive analysis of allo (in which no projectsthe full portfolio (whichld be impractical even ifexample, a small sample

approaches (for the same xed budget) would always be the same ifall projects were not only independent but also completely divisiblewith constant returns to scale [65, p. 151] (see also [21]). This is nottrue when the projects are indivisible, as is assumed in this paper(see Section 2), or when projects interdependences and other typesof constraints are present (see Section 3.2). Portfolios selectedthrough the prioritization approach exclude any project with abenet-to-cost ratio that is lower (that is, a less productive or prof-itable project [13,19]) than the benet-to-cost ratio of an unselectedproject. The same is not necessarily true of portfolios selected through1. Introduction It is well known that the portfolio selected by each of these twoPROBEA multicriteria decision support s

Joo Carlos Loureno a,, Alec Morton b, Carlos A. Bana Centre for Management Studies of Instituto Superior Tcnico (CEG-IST), Technical Universb Management Science Group, Department of Management, London School of Economics, L

a b s t r a c ta r t i c l e i n f o

Article history:Received 11 August 2009Received in revised form 30 June 2012Accepted 3 August 2012Available online 17 August 2012

Keywords:Portfolio decision analysisResource allocationPortfolio robustnessRestricted efciencyDSS

This paper addresses the promultiple criteria, different prered an undoubtedly robustthe projects there is no othebenet.Wepresent a newdeit implements. PROBE identiportfolio cost range, and peThe robustness evaluation sand differences in project comthe proposed portfolio instea

j ourna l homepage: wwwould be found by solvingblem [33,48] that maxi-g the budget constraint

+351 214233568.no), a.morton@lse.ac.uk

rights reserved.stem for portfolio robustness evaluation

e Costa a

of Lisbon, Av. Rovisco Pais 1, 1049-001 Lisbon, Portugaln WC2A 2AE, United Kingdom

m of selecting a robust portfolio of projects in the context of limited resources,ct interactions and several types of uncertainty. A portfolio of projects is consid-ice if for a given uncertainty domain that affects the costs and/or the benets ofortfolio that does not cost more and simultaneously may provide more overallon support system, PROBE (Portfolio Robustness Evaluation), and the algorithmsall efcient portfolios and depicts the respective Pareto frontier within a givents users to analyze, in depth, the robustness of selecting a proposed portfolio.s by identifying competitor portfolios to the proposed portfolio, its similaritiessition to its competitors, and the regret a decision-makermay have by selectingf a competitor.

2012 Elsevier B.V. All rights reserved.

ort Systems

l sev ie r .com/ locate /dssfolio [44], and Expert Choice Resource Aligner [25]. Of these, Equityand HiPriority follow the prioritization approach, Expert ChoiceResource Aligner follows the optimization approach, while LogicalDecisions Portfolio implements both approaches.

Section 3 introduces PROBE (Portfolio Robustness Evaluation), anew decision support system for multicriteria portfolio analysis thatimplements the optimization approach and also nds the solutionsgiven by the prioritization approach. When several benet criteria

project j (j=1,,m). For simplicity, without loss of generality, also as-sume that them projects of X are presented in decreasing order by theirbenet-to-cost ratios such that rjrj+1, j=1,, m 1, as the fourprojects (1, 2, 3, and 4) in Table 1 are.

In the ensuing, we will frequently suppose that the value ofprojects is given by a multiattribute value model. In this case, let vijbe the value score of project j on the benet criterion i, i=1,, n(n1) and wi (wi0) the weight of criterion i, i=1,, n (with

535J.C. Loureno et al. / Decision Support Systems 54 (2012) 534550are dened, PROBE calculates the benet value of each projectthrough an additive value model. Therefore, the basic project inputsfor a multicriteria portfolio analysis are each project's cost and valuescores on the benet criteria, and the weights that capture trade-offs between criteria (see Section 3.1). For a resource allocationmodel dened with the data inputted by the user, PROBE identiesall efcient portfolios and depicts the respective Pareto frontierdistinguishing the convex from the non-convex efcient portfolios,through the algorithms presented in Section 4.1. Various types ofconstraints can also be incorporated (see Section 3.2) to account forproject interactions or interdependencies and other programmaticissues, althoughmodel builders should be careful because a temptationfor managers is to use constraints to protect existing spend and hencemore and more constraints keep an organisation pumping resourcesinto the status quo, thereby preventing the organisation from movingin new strategic directions [56, p. 55].

In real-world resource allocation contexts, several sources ofuncertainty can affect the precision of some of the model's inputs.Often, a best portfolio is selected on the basis of best guess inputdata only. Therefore, to avoid the trap of false precision [62] it iswise to evaluate the robustness of the best portfolio by simultaneouslyconsidering data uncertainties affecting the costs and benets of theportfolios (i.e., imprecise project costs, project benet values and criteriaweights). Portfolio robustness evaluation (see Section 4.2) has been thecore motivation for the conception of PROBE. Our above mentionedstudy of commercial packages for multicriteria resource allocation andportfolio selection [46] revealed that none of them permit users toperform an a posteriori sensitivity analysis on several inputs simulta-neously. The example in Section 5 illustrates how this type of robust-ness analysis can be conducted with the DSS PROBE within a givenuncertainty domain, and nally the example in Section 6 describesbriey an application of PROBE to health service planning in NorthernLisbon, which demonstrates the usefulness of the software and theapproach. This new DSS extends the original PROBE software [2,9](which was limited to the preference robustness evaluation of projects)to portfolio decision analysis.

Pioneering work on robustness in multicriteria portfolio analysiswas conducted by J. Liesi, P. Mild and A. Salo [42,43] in the develop-ment of RPM, Robust Portfolio Modeling, rstly outlined in [61] andimplemented in the non-commercial software RPM-Decisions (http://www.rpm.tkk./rpm-software.html). However, contrary to PROBE,the multicriteria decision-aid provided by RPM is not concerned withanalyzing the stability [12] of a solution (or selection). The idea of ro-bustness analysis shared by RPM turns the ex-post sensitivity analysisperspective upside down [59], by incorporating uncertainty a priorias incomplete information in the formulation of the problem andlooking for good portfolios: This incomplete information is modeledthrough sets of feasible parameter values and decision recommenda-tions are given based on the computation of non-dominated portfolios[41, p.12]. There is therefore a basic procedural difference between therobustness approaches of RPM and PROBE (see Section 4.2).

2. Basic concepts and portfolio selection approaches

In this paper, portfolio selection is only concerned with a set X={j : j=1,,m} ofm projects that areworth funding; it assumes that pro-ject proposals that are not worth funding were screened out in a previ-ous phase of the selection process. Conceptually, the benet value of aproject that does not add value to a portfolio should be zero; conse-quently, the benet value of a project that is worth funding should bedened as the value that the project adds to the portfolio [50]. Letcj>0 and vj>0 represent, respectively, the cost and the benet valueof project j of X and B the budget available, and assume there is nocost associated with not funding project j (j=1,,m) [18]. (Situationswhere there are costs associated with not selecting projects are

discussed in Section 3.1). Let rj=vj/cj be the benet-to-cost ratio ofni1wi 1). The benet value vj of project j is given by

vj ni1wivij: 1

We will now introduce some concepts which are standard in theliterature [24,60], but which we formally dene here in our context.

Denition 1. A portfolio p is a subset of projects of X (ppX). It maybe that not all combinations of projects are possible as portfolios(e.g. expand service S and contract service S cannot be donesimultaneously as a matter of simple logic) and so their numbermay be less than 2m (see Section 3.2 for more details).

Let cp and vp be the cost and the benet of portfolio p given by,respectively

cp jpcj 2

and

vp jpvj jpni1wivij: 3

Denition 2. A portfolio p dominates another portfolio d if cpcd andvp>vd, or if cpbcd and vpvd. A portfolio is efcient (Pareto-efcient,Pareto-optimal or non-dominated)whennoother portfolio dominates it.

Fig. 1 shows all (16=24) portfolios that can be formed with thefour projects of Table 1 (including the empty portfolio {}). There areseven efcient portfolios, shown as squared dots in Fig. 1; they formthe efcient or Pareto frontier.

Denition 3. An efcient portfolio p is a convex efcient portfolio if,and only if, there exists a real number u]0,1[ such that for everyportfolio d with at least one of vdvp and cdcp, uvp(1u)cp>uvd(1u)cd.

Denition 4. An efcient portfolio p is a non-convex efcient portfolioif, and only if, there exist two other efcient portfolios l and h withcosts cl and ch, and benets vl and vh, and a ]0,1[ such that(i) cpch+(1)cl and (ii) vpvh+(1)vl with at least oneof these inequalities strict (that is, p is dominated by a linear combi-nation of l and h).

Proposition 1. No efcient portfolio can be both convex efcient andnon-convex efcient.

Proof. Suppose such a portfolio existed. Then it must be the casethat there exist efcient portfolios l and h, and a ]0,1[ such thatcpch+(1)cl (4) and vpvh+(1)vl (5) with at leastone inequality strict, and moreover, for some real u]0,1[, uvp(1u)cp>uvh(1u)ch (6) and uvp(1u)cp>uvl(1u)cl (7).

Table 1Benet values, costs and benet-to-cost ratios of four projects.

Projects j vj cj vj/cj

1 3 4 0.752 4 8 0.503 3 10 0.304 2 8 0.25

benet), but it can also be easily seen from the observation of theprevious paragraph that the non-convex efcient portfolios will beomitted, hence the only portfolios formed by this method are convexefcient ones. Hence, in the prioritization approach the notion ofvalue-for-money of a project [7,55,56] or its bang-for-the-buck[13,16,20] is associated with the slope of each project's benet-to-cost triangle, as shown in Fig. 2 for the four projects of Table 1. Thelast column of Table 1 shows that the order of selection by prioritiza-tion would be: rst project 1, then project 2, followed by project 3,and nally project 4. When the budget increases from 0 to 30 (seeFig. 2), the sequence of portfolios selected through the prioritization,from the empty portfolio {} to the full portfolio {1, 2, 3, 4}, starts withportfolio {1} for 4Bb12, followed by portfolio {1, 2} for 12Bb22,and then portfolio {1, 2, 3} for 22Bb30 (therefore ignoring thenon-convex efcient portfolios {2} and {1,2,4}).

Alternatively, the portfolio selected by the optimization approach

536 J.C. Loureno et al. / Decision Support Systems 54 (2012) 534550Multiplying Eqs. (4) and (5) by (1u) and u respectively gives (1u)cp(1u)ch(1)(1u)cl and uvpuvh+(1)uvlwith one inequality strict, and adding these, and combining withEqs. (6) and (7) and simplifying gives both uvl(1u)cl>uvh(1u)ch and uvl(1u)clbuvh(1u)chwhich is a contradiction.

Denitions 2, 3 and 4 illustrate the managerial arguments in favorof or against selecting convex versus non-convex efcient portfolios.If a decision-maker feels that there is some approximately constantmarginal value of money (because she has a good idea of the alterna-tive uses to which unused funds can be put within her organization),so that expenditure c associated with a portfolio (suitably scaled) canbe deducted from the benet-value v of that portfolio to come upwith an overall index uv(1u)c of portfolio attractiveness, thenshe would restrict her focus to convex efcient portfolios as denedin Denition 3. If on the other hand, the decision-maker has no suchconception of marginal value for moneyperhaps because money ismade available on an use it or lose it basis, and if unspent cannotbe diverted to other worthwhile purposes [34, p. 205]then any ef-cient portfolio (even those which are non-convex efcient accordingto Denition 4) maximizes value within some budget constraint (byDenition 2), and so may be a contender for selection.

In Fig. 1, {}, {1}, {1, 2}, {1, 2, 3}, and {1, 2, 3, 4} are convex efcientportfolios (they form the convex efcient frontier), and {2} and {1, 2,4} are non-convex efcient portfolios.

Non-convex efciency has the following interesting interpretation.For any non-convex efcient portfolio p, we can suppose without lossof generality that clbcpbch and vlbvpbvh (for each of h and l, eitherthe costs are less or the benets are greater than those of p, otherwisethe inequalities of Denition 4 cannot hold, and if, for either of h or l,both the costs are less and the benets are greater, then the efciencyof p is contradicted). Rearranging (i) and (ii) from Denition 4 givescpcl(chcl) and vpvl(vhvl) (with at least one inequality

0

2

4

6

8

10

12

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30

Cum

ula

tive

ben

efit

Cumulative cost

{ }

{1, 2}

{1, 2, 3}

{1, 2, 3, 4}

{1}{2}

{1, 2, 4}

Fig. 1. Chart showing the portfolios that can be formed with the four projects. Efcientportfolios are represented by squared dots and dominated portfolios by triangular dots.The composition of each portfolio is shown in brackets next to the corresponding dot.strict) and dividing the latter inequality by the former gives(vhvl)/(chcl)>(vpvl)/(cpcl), which is to say, the marginal benet ofexchanging h for l is always greater than the marginal benet ofexchanging p for l. By a similar reasoning (chcp)/(vhvp)b(chcl)/(vhvl) always holds, by denition, whatever constraints are present;i.e. the marginal cost of an additional benet unit when selecting hinstead of p is always lower than the marginal cost of an additionalbenet unit when selecting h instead of l, which may be a relevantpiece of information for decision-making.

Given a xed budget B, the prioritization approach selects theportfolio formed by the projects j, j=1,, k with km, such thatkj1cjB andk1j1 cj > B. This approach implicitly assumes besidesthe budget constraints no other constraints are binding and that allcombinations of projects are possible. It can be easily seen that port-folios built in this way for increasing values B will be efcient (be-cause in such a portfolio substituting a project jk with a set ofprojects with lesser total cost can only lead to a reduction in overallis the optimal solution of the following binary integer programmingproblem (known as the 01 knapsack problem [47]):

maximize mj1vjxj;subject to : mj1cjxj B;

xj 0;1f g; j 1;;m;8

where xj is a binary variable such that xj=1 if project j is in theoptimal portfolio and xj=0 otherwise.

For the four projects in Table 1 and a budget B=20, the optimalportfolio is {1, 2, 4} with a benet of 9 for a cost of 20 (see Fig. 1),whereas the portfolio selected by the prioritization approach, forthe same budget, would be {1, 2} with a benet of 7 for a cost of 12(see Fig. 2). Fig. 1 shows that both portfolios are efcient, but itseems that optimization identies a better portfolio than prioritization,in the sense that {1, 2, 4} is a higher benet portfolio which is neverthe-less still affordable. However, note that portfolio {1, 2, 4} includesproject 4, which is less productive than the non-selected project 3(because r4b r3; see Table 1), and in this sense one could argue that{1, 2} is a more attractive portfolio.

It is also important to stress that often managers are not onlyinterested in nding the best solution for a specied resource amountbut also in exploring a budget band, namely when a xed budget isnot known at the time of the analysis or is expected to change. Inthese cases, a conservative argument favors resource allocation pro-cesses by order of priority, because additional resources are allocatedto projects not yet selected without removing previously selectedprojects from the best portfolio, whereas using the optimizationapproach may disrupt previous selections. For example, for a budgetrange of 202, in Fig. 1, portfolio {1, 2} is optimal for 18bBb20, thenproject 4 enters the optimal portfolio when 20Bb22, but for B=22project 4 is replaced by project 3. This is because the optimization

0

2

4

6

8

10

12

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30

Cum

ulat

ive

bene

fit

Cumulative cost

1

2

3

4

12

34

{1, 2}

{1, 2, 3}

{ }

{1, 2, 3, 4}

1

2

3

4

{1, 2}

{1, 2, 3}

{1, 2, 3, 4}

1

2

3

4

{1, 2}

{1, 2, 3}

{1, 2, 3, 4}

{1}

value-for-money slopes

Fig. 2. Cumulative cost versus cumulative benet chart showing the portfolios formedby the benet-to-cost ratio approach. The value-for-money of each project is given bythe slope of its benet-to-cost triangle. The arrow in the value-for-money slopes box

shows the direction of improvement of the benet-to-cost value of the projects.

537J.C. Loureno et al. / Decision Support Systems 54 (2012) 534550approach may select any portfolio that is efcient, whereas the prioriti-zation approach will only select portfolios that are convex efcient.When the optimal solution of the knapsack problem (8) is a convexefcient portfolio, the portfolio selected by prioritization is the same,but, when the optimal solution of problem (8) is a non-convex efcientportfolio, the portfolio selected by prioritization is the rst convex ef-cient portfolio at its left in the convex efcient frontier. This portfoliocould be found by constraining the knapsack problem in such a waythat the optimal solution does not include any project with a lowerbenet-to-cost ratio than a non-selected project. (The prioritizationprocedure is also known as the greedy algorithm for the knapsackproblem [33,39]).

The conservative property of convex efcient portfolios allows forthe denition of a funding strategy by order of priority of the projectsthat is independent of the funding constraint so that the entirefunding decision does not have to be revised every time the fundingconstraint changes [15], as it is often desired in both public andprivate organizational decision contexts (e.g. the case-studies describedin [7] and [56] respectively). Unfortunately, this property can be lostwhen in the presence of project interactions or other constraints (seeSection 3.2). Regardless, in this sense convex efcient portfolios areless volatile than non-convex efcient ones; nevertheless, we considerthat this does not justify showing only the former to the decision-makers, while hiding the latter, nor adopting a heuristic project prior-itization [38].

Finally, it is worthwhile to raise a technical issue. If there are atleast two projects with the same benet-to-cost ratio, some convexefcient portfolios are not identied by the prioritization approach.For example, if there are only two projects a and b with the samebenet-to-cost ratios the prioritization approach either selects port-folio {a} followed by portfolio {a, b}therefore missing portfolio{b}or it selects portfolio {b} followed by portfolio {a, b}thereforemissing portfolio {a}. The prioritization approach can, however, easilydeal with a large number of projects, contrary to knapsack optimizationalgorithms. Indeed, the knapsack problem (8) is technically difcult tosolve despite its straightforward structure, due to the integrality con-straints xj{0,1}, j=1,, m. The knapsack problem is considered tobe a nondeterministic polynomial-time hard (NP-hard) problem [26](a signicant number of exact and approximate resolution algorithmsfor this problem have been thoroughly studied [33,48]).

3. Introducing the DSS PROBE

3.1. The MCDA and PDA components and basic input data

PROBE is a multicriteria decision support system for portfoliorobustness evaluation that integrates two main architectural compo-nents: a multicriteria decision analysis (MCDA) component and aportfolio decision analysis (PDA) component.

The MCDA component allows the user to structure the benetcriteria in the formof a value tree and input data for the costs of the pro-jects and their benet scores on each bottom-level criterion of the valuetree. Let X be a specic set of projects j (j=1,,m) dened by the user.Even when uncertainty is present, PROBE always asks the user to input,for each project j, a (best guess) cost cj and (best guess) benetvalue scores vij on each bottom-level criterion i, i=1,, n (n=1 ifonly one benet dimension, such as NPV, is dened). For a value treewith only one level of n>1 benet criteria i (i=1,, n), (bestguess) weights wi (i=1,, n) should be introduced and PROBE com-putes the benet value vj of each project j (j=1,, m) by applyingthe non-hierarchical additive model (1). If the value tree has two ormore levels below the root node, specic weights are dened for thecriteria at each level and PROBE uses a hierarchical valuemodel to com-pute an aggregate benet value vj for each project j (j=1,,m) by ap-plying model (1) bottom-up successively. If a branch of risk criteria is

included in the value tree set of criteria, the vj of each project j is moreadequately designated by risk-adjusted benet [56]. For the sake ofsimplicity, without loss of generality, all programs and algorithmspresented in this paper assume a non-hierarchical benet model,which can be easily extended to the corresponding generic hierarchicalformulation implemented in PROBE.

For the given positive project costs cj and benet scores vj (j=1,,m), the PDA component solves the knapsack optimization problem(8)with or without additional linear constraints added by the userto model project interactions (see Section 3.2)for any xed budgetB, nds all efcient portfolios, distinguishes convex from non-convexones (see Section 4.1)a functionality not included in the softwarepackages analyzed in [46]and displays the portion of the Paretofrontier for a user-dened limited portfolio cost range XB; B

(see

Section 4.1). When the number of projects of X is compatible with areasonable computational time (see Appendix A), PROBE can displaythe full efcient frontier, assuming by default XB=0 and B mj1cj.

If there are costs associated with not selecting projects, the totalcost of the projects not selected should be subtracted from thebudget. This can be modeled by replacing in problem (8) the budgetconstraintmj1cjxj B bymj1cjxj Bmj1c0j 1xj

where cj0 is

the cost of not selecting project j, which is equivalent to

mj1 cjxj c0j 1xj h i

B (see [18,35]).Concerning the modeling of uncertainty, PROBE allows the user to

input: a set c of plausible cost ranges j1;;m

Xcj; cjh i

such that Xcj

cj cj (j=1,, m); a set v of plausible benet scores ranges

j1;;m

i1;;nXvij; vijh i

such that Xvij vij vij (i=1,, n; j=1,, m);

and a system of (non-strict) linear inequalities or equalities on theweights (e.g. weights rankings and/or weights ranges) dening a poly-hedronw of feasible weights such thatwRw. Observe that as the in-equalities are non-strict w is closed and as the weights are byassumption non-negative and sum to unity w is bounded, hence wis compact. The MCDA component uses the additive model to calculate

by optimization the feasible benet value range Xvj; vjh i

dened by v

and w for each project j (j=1,, m) as follows: Xvj minwRw

ni1wiXvijand vj max

wRwni1wivij. It is within a user-dened uncertainty domain

, comprisingc,v andw, that portfolio robustness evaluation takesplace (see Section 4.2).

PROBE is coded in the C++programming language [63], using theC++ Builder development software having Microsoft Windows as itstarget environment. PROBE requires the user: to create a value tree byadding criteria nodes, which is graphically shown as an inverted treewhere the user can easily drag-and-drop nodes; to input criteriaweights and to input the data concerning the projects (names, costsand benet value scores on each of the bottom-level criteria).PROBE reads and writes its own data les and can easily exchangepieces of information between its visual components or between itsvisual components and other programs using the Windows clipboardwith cut-and-paste and copy-and-paste functionalities. PROBE usesthe mixed integer linear programming (MILP) solver lp_solve 5.5.2.0(available at http://sourceforge.net/projects/lpsolve/) to solve theoptimization problems included in the PROBE algorithms presentedin Section 4. This solver is based on the revised simplex method andon the branch-and-bound algorithm to deal with integer variables.Despite being a user-friendly decision support system, PROBE requiresusers to have some knowledge of portfolio decision analysis.

3.2. Inputting data for modeling project interactions

3.2.1. Synergies among projectsTo address a synergy between the costs and/or the benets of twoprojects s and t, an auxiliary project s,t must be added to X together

538 J.C. Loureno et al. / Decision Support Systems 54 (2012) 534550with the synergistic effect on cost cs,t and the synergistic effect on thebenet values vi,s,t on each benet criterion i, i=1,,n. PROBE thenautomatically denes an extra binary variable xs,t such that xs,t=1 ifboth projects are selected for the portfolio, or xs,t=0 otherwise, andadds the following three constraints to problem (8):

xs;txs 0;xs;txt 0;xs xtxs;t 1:

9

Synergies between more than two projects imply adding moreconstraints. Let us see an example with three projects s, t, and uthat synergize only when the three of them are simultaneouslyincluded in a portfolio (and they do not synergize when only twoare in the portfolio). In this case, an auxiliary project s,t,u must beadded to X, together with their synergistic effect on cost cs,t,u and onthe benet values vi,s,t,u on each benet criterion i, i=1,,n. PROBEthen denes an additional binary variable xs,t,u such that xs,t,u=1 ifthe three projects are selected for the portfolio, or xs,t,u=0 otherwise,and adds the following four additional constraints to problem (8):

xs;t;uxs 0;xs;t;uxt 0;xs;t;uxu 0;xs xt xuxs;t;u 2:

10

3.2.2. Constraints on projectsBesides synergy effects, PROBE also allows the user to add other

well known types of constraints to problem (8), to model differenttypes of project interactions, such as: include project j, xj=1; excludeproject j, xj=0; dependency between two projects i and j (i can onlybe selected if j is also selected), xixj0; any portfolio including imust also include j and vice versa, xixj=0; mutual exclusivity oftwo projects i and j, xi+xj1; group constraints on a subset G ofmG projects (1mGm), such as (with 0qmG), jGxj q;jGxjq; andjGxjq:

PROBE includes an interface that allows the user to add to problem(8) any other type of linear constraints [14,27,30], e.g. to tacklemulti-period budgeting problems. A feasible portfolio is a portfoliothat respects all of the constraints introduced by the user.

Henceforth, we will write xP to indicate that x is a member of asubset of {0,1}m dened by a family of constraints of the typediscussed in this section. We suppose that P captures the set of all01 vectors corresponding to possible portfolios.

4. PROBE innovative functionalities

4.1. Finding all efcient portfolios within a given portfolio cost range

For project costs cj and benet scores vj (j=1,, m) given by theMCDA component, and supposing for now that no project interactionconstraints were dened, the PDA component starts searching for theefcient portfolios, within a given portfolio cost range XB; B

, by solv-

ing problem (8) with B B: Next, problem (8) is again solved with Bequal to the cost of the optimal portfolio previously found minus asmall enough amount , and so on while B XB. The algorithmdesigned to implement this process, FindEfcientPortfolios, presented inFig. 3, is also capable of identifying all possiblemultiple optimal solutions.Finally, PROBE uses another algorithm, FindConvexEfcientPortfolios,presented in Fig. 4, to differentiate convex efcient from non-convexefcient portfolios. Additional linear constraints of the types describedin Section 3.2 can easily be added to algorithm FindEfcientPortfolios totake project interactions into account when nding efcient portfolios.We make the following observations.Proposition 2. With sufciently small, the algorithm FindEfcientPortfoliosnds all and only the efcient portfolios within a given portfolio cost range

XB; B

.

Proof. The algorithm makes repeated calls to the optimization prob-lem opt1, and nds optimal portfolios x* and writes them and theassociated benets and costs to a matrix mEP. By the optimality ofx*, the only way x* can fail to be efcient is if there is a portfoliowith the same benet but lower cost: however such an x* wouldhave been be deleted from mEP by the corresponding while state-ment. Hence, at the termination of the algorithm, the only entries inmEP will be the efcient portfolios along with their benets and costs.It remains to show that the algorithm nds all efcient portfolios(given sufciently small ). To see this, denote the set of all efcientportfolios with costs in the range XB; B

by E. Without loss of generality

index the portfolios in E as p1,,pk such that c(p1)c(pk), wherev(p1),,v(pk) and c(p1),,c(pk) are the total benets and costs associ-atedwith these portfolios. Observe that by the assumption of efciency,v(p1)v(pk) and moreover that for any pair of successive efcientportfolios i,i+1:c(pi)=c(pi+1), then v(pi)=v(pi+1). Partition E intosubsetswith portfolios pand p in the same subset if and only if the cor-responding costs and benet values are equal. Denote these subsets asE1,, Eqwhere the indices of the subsets are congruentwith the indicesof the contained portfolios (i.e. higher indexed portfolios belong tohigher indexed subsets). If the algorithm nds a portfolio in some sub-set Ei, the duplicate checking subroutinewill ensure that all such portfo-lios are written to mEP. The reader will note that the NRS constraints

mj1xjxj mj1xj1n o

ensure that this subroutine does not cycle

and so terminates in nite time. We can observe that appending thisconstraint to opt1will only eliminate, besides x*, portfolios which con-tain x* and so which could not have been feasible at the previous itera-tion (since, beinghigher value than x*, theywould have been an optimalsolution at this iteration). Once the algorithm has exhausted the equiv-alence class Ei, opt1will become infeasible, and the else if statementwillbe invoked; assuming is sufciently small, the next solution of opt1will be a member of Ei+1. These remarks, combined with the observa-tion that on initialization, the rst solution of opt1 will nd a memberof the equivalence class E1, demonstrate the truth of Proposition 2.

Proposition 3. The algorithm FindConvexEfcientPortfolios nds alland only the convex efcient portfolios within a given set of efcientportfolios.

Proof. The algorithm operates on the matrix mEP, of costbenet pairsof all efcient portfolios, which at the conclusion of FindEfcientPortfolioshas been sorted by increasing order of cost. By Proposition 1, everyefcient portfolio can be classied as either convex efcient ornon-convex efcient. To see that the algorithm nds convex efcientportfolios, consider the following. The maxSlope generated at theearlier iterations will be nonstrictly greater than the later iterations.To see this, suppose it is not the case that the slopes strictly declinebetween two iterations. Then there must be points in the cost benetspace associated with portfolios i1, i, i+1: (ci1,vi1), (ci,vi),(ci+1,vi+1) picked out by the algorithm in successionwith ci1bcibci+1

and vi1

bvibvi+1 (inequalities can be assumed to be strict because byconstruction points correspond to distinct efcient portfolios) and(vi+1vi)/(ci+1ci)>(vivi1)/(cici1). Because we know thesign of the denominators we can cross-multiply and simplifyinggives vi+1ci+vici1+vi1ci+1>vici+1+vi1ci+vi+1ci1. However,it must also be the case that (vi+1vi1)/(ci+1ci1)(vivi1)/(cici1) (otherwise the algorithm would have skipped i) and we canlikewise cross-multiply that to get vi+1ci+vici1+vi1ci+1vici+1+vi1ci+vi+1ci1, and so we have derived a contradiction, so the slopesbetween consecutive points do indeed nonstrictly decline we will usethis fact later on, so call it (A). We also observe that since slopes between

l lconsecutive points decline, then if we have any three points (c ,v ),

539J.C. Loureno et al. / Decision Support Systems 54 (2012) 534550(cp,vp), (ch,vh) picked out by the algorithm with clcpch andvlvpvh, then it must be the case that (vhvp)/(chcp)>(vhvl)/(chcl) (this is an consequence of the elementary arithmetic observationthatwith a, b, c, d all positive, c/dba/bbcbad bc+cdbad+cdc/db(a+c)/(b+d)). We call this fact (B).

Now the proof is by induction. First we establish the base case. Therst point (i.e. the cost and benet associated with the rst portfolio)(c0,v0) must be convex efcient (e.g. by checking the denition ofnon-convex efciency: it obviously cannot be non-convex efcient

Fig. 3. Algorithm Findand so by Proposition 1 must be convex efcient). To deal with theinductive case we suppose that all points found by the algorithm upto the ith point (which we denote as (ci,vi)) have been demonstratedconvex efcient. We now proceed to show that the (i+1)th point(ci+1,vi+1) is also convex efcient, which we do by demonstratingthat 1/[1+(vi+1vi)/(ci+1ci)] is a value of u as specied inDenition 3. If this number does not fulll the conditions ofDenition 3, there exists a point (c*,v*) such that v* [(vi+1vi)/(ci+1ci)]c*>vi+1 [(vi+1vi)/(ci+1ci)]ci+1 (11). We will

EfcientPortfolios.

540 J.C. Loureno et al. / Decision Support Systems 54 (2012) 534550assume Eq. (11) holds and seek a contradiction. We know that c*cibecause then v*=vi, as noted in the proof of the previous proposition,and so we can restrict our attention to two cases.

Case 1. If c*>ci, we can multiply Eq. (11) through by (ci+1ci) toget v*ci+1+vic*+vi+1ci> vi+1c*+vici+1+v*ci. However, we knowthat (ci+1,vi+1) was chosen to have maximal slope from (ci,vi) and so(v*vi)/(c*ci)(vi+1vi)/(ci+1ci). Both denominators are posi-tive, sowe can crossmultiply and get v*ci+1+vic*+vi+1civi+1c*+vici+1+v*ci, sowe have a contradiction andwe can conclude that thereexists no such point (c*,v*) and (ci+1,vi+1) is convex efcient.

Case 2. If c*bci, there must be a k, k+1with k+1 i: ckbc*bck+1 andvkbv*bvk+1. Fact (B) tells us that (vi+1vi)/(ci+1ci)b(vi+1vk+1)/(ci+1ck+1). Rearranging this give us vk+1[(vi+1vi)/(ci+1ci)]ck+1bvi+1 [(vi+1vi)/(ci+1ci)]ci+1 and combiningthis with Eq. (11) gives us v* [(vi+1vi)/(ci+1ci)]c*>vk+1[(vi+1vi)/(ci+1ci)]ck+1 (12). Now, we know from the inductivehypothesis that v* [(vk+1vk)/(ck+1ck)]c*vk+1 [(vk+1vk)/(ck+1ck)]ck+1 (13). We can subtract Eq. (12) from Eq. (13)to get c*[(vk+1vk)/(ck+1ck)(vi+1vi)/(ci+1ci)]>ck+1[(vk+1vk)/(ck+1ck)(vi+1vi)/(ci+1ci)]. We are nowgoing to use fact (A) which tells us that (because the difference in ra-tios which appears on both sides is positive) we can divide both sidesand the direction of the inequality remains unchanged, so we getc*>ck+1. But this contradicts the denition of ck+1, so we rejectthe existence of (c*,v*).

Fig. 4. Algorithm FindConFrom inspection of these two cases, we conclude that (ci+1,vi+1)is optimized by the line v [(vi+1vi)/(ci+1ci)]c, hence it is con-vex efcient.

To see that the algorithm nds all convex efcient portfolios,suppose that we have convex efcient portfolios l, p, h: clbcpbch

and vlbvpbvh and the algorithm skips p. Then, for the algorithm toskip p, it must be the case that (vhvl)/(chcl)>(vpvl)/(cpcl).But clbcpbch we can nd o :cp=och+(1o)cl. Substituting thisinto the inequality on the gradients, cross-multiplying and simplifyingyields vpbovh+(1o)vl, so p is non-convex efcient by Denition 4which is a contradiction.

4.2. Portfolio robustness evaluation

For the given costs cj and benet scores vj (j=1,, m) of theprojects, let p*, with cost cp and benet vp, be a specic efcientportfolio selected by the user, either by asking PROBE to nd theoptimal solution of problem (8) for a xed budget B or by inspectionof the efcient portfolios found by PROBE within a portfolio costrange XB; B

. In one situation or the other, the user may be concerned

with the robustness of the choice of p* in the face of an uncertaintydomain (comprising c, v and w, see Section 3.1), withinwhich Xc

pcpcp and Xvpvpvp.

Denition 5. Given an uncertainty domain, we say that portfolio p isrestricted efcient relative to p* if and only if (i) jp0 5pXcjjp5p0cj

vexEfcientPortfolios.

and (ii) there is a combination of feasible weightsw and of feasible ben-et value scores v such thatni1wi jp5pvijjp 5pvij

b 0.

The motivation for this restricted efciency concept is thathavingformed the expectation that he will purchase p*the decision-makeris only interested in portfolios which may cost less than p* and ofthese possibly cheaper portfolios he is only interested in those whichmay also be better than p*. This differs from the standard efciency ornon-dominated concept of multiobjective programming, in which therelevant solution concept is the set of all portfolios which could eitherbe better or be cheaper than p*. However, that solution concept isboth well-studied and, more importantly, not relevant in our context,since large numbers of irrelevant portfolios would qualify as efcientor non-dominated in that standard sense (e.g. the empty portfoliocontaining no projects).

Using algorithm FindCandidates, described in Fig. 5, PROBE identiesa set of portfolioswhich contains the set of restricted efcient portfolios.Next, PROBE uses algorithm FindRestEfcientPortfolios, described inFig. 6, to nd which candidate portfolios p are restricted efcient rela-tive to the proposed portfolio p*.

Proposition 4. ThealgorithmsFindCandidates andFindRestEfcientPortfoliosnd all and only the portfolios which are restricted efcient relative to p*.

Proof. First we show that FindCandidates nds all the restricted ef-cient portfolios relative to p*. According to Denition 5, conditionalefciency has two parts, a cost condition (i) and a benet condition(ii). Using the notation of the algorithm, observe that for any portfoliop which fullls condition (ii), by denition there exist wo and vo

541J.C. Loureno et al. / Decision Support Systems 54 (2012) 534550Fig. 5. Algorithm FindCandidatePortfolios.

dRe

542 J.C. Loureno et al. / Decision Support Systems 54 (2012) 534550withni1woi jp5pvoijjp 5pvoij

b 0, and, because by optimality

Fig. 6. Algorithm Finjp 5pvj ni1woijp 5pvoij and ni1woijp5pvoij Xvp

jppvj, it is also the case that jp 5pvj > Xvp

jppvj,

although the reverse implication does not hold. Call jp 5pvj >Xvpjpp

vj, condition (ii). Hence, if we can enumerate all portfoli-

os which fulll (i) and (ii), we have a superset of the restricted efcientportfolios. Associate with each portfolio p in this superset a valuation ofp according to the functionjp0 5pvj jp0p

vj and call this valua-

tion v(p). This gives us an ordering on the superset (not necessarilystrict). FindCandidates proceeds down this list from high to low v(p)through successively solving opt3. The constraint set ensures condition(i) is met by the constraintjpXcjxj jpcjxjcp. p* itself is elim-inated by the constraintmj1xjxjmj1xj1. Cycling is prevented bymeans of the NRS constraints, and growth in the NRS constraint set ismanaged by tightening the bound vS which makes at least one of con-straints in NRS redundant. The NRS constraints also ensure that wherethere aremultiple portfolios with equal valuations, all will be enumerat-ed, in amanner similar to that of FindEfcientPortfolios (aswas discussedin the proof of Proposition 2). The termination criterion ensures thatcondition (ii) is fullled. Hence, FindCandidates nds all restricted ef-cient portfolios (as well, perhaps, as some others). The algorithmFindRestEfcientPortfolios directly checks each portfolio p ' found byFindCandidates against p* using Denition 5 in order to establish wheth-er p' is restricted efcient, thus ensuring that only restricted efcientportfolios (p) are retained.

The computational time for robustness evaluation depends on thenumber of projects taken into account and on the uncertaintydomain dened (the effects of the uncertainty on the computa-tional times using trials with 50, 75 and 100 projects are shown inAppendix A).We say that the choice of p* is undoubtedly robust when no

stEfcientPortfolios.portfolio p exists which is restricted efcient relative to p*. Other-wise, for each restricted efcient portfolio p PROBE solves problem(14)

z : max ni1 wi jp5p vijjp 5pXvij

subject to : wRw: 14

The result z is the upper bound of the range of variation of thedifference between the benet value of the proposed portfolio p*and the benet value of the competitor portfolio p in the uncer-tainty domain , with the lower bound Xz calculated by algorithmFindRestEfcientPortfolios (see Fig. 6). Finally, using expressions (15)and (16), PROBE calculates the lower bound Xd and upper bound dof the range of variation of the difference between the costs of p*and p in .

Xd jp 5pXcjjp5pcj 15

d jp 5pcjjp5pXcj 16

The user is then able to analyze the differences in cost and benetbetween the proposed portfolio p* and each competitor portfolio p,as illustrated with an example in Section 5. Finally, core projectscan be identied as those projects that are common to the proposedportfolio p* and all of its competitors.

5. Example

In this section we present an example, which has been constructedto illustrate the key functionality of PROBE. A manager has to allocateresources to 12 projects from four departments, totaling 14 million

Table 2Basic input data for the 12 projects and MCDA output.

Project Dept. Cost (106) Benet value scores Benet value

543J.C. Loureno et al. / Decision Support Systems 54 (2012) 534550(see Table 2), greatly exceeding the 5 million available. The managerand the four department directors constitute the decision makinggroup (DMG) that develops a portfolio decision analysis with the sup-port of PROBE. The DMG evaluates the added value of each project oneach one of four benet criteria (B1, B2, B3 and B4). The respective ben-et value scores of the projects are shown in columns B1 to B4 inTable 2. Using this data and the criteria weights indicated in the lastrow of Table 2, the MCDA component of PROBE computes the benetvalues of the projects shown in the last column of Table 2.

Fig. 7 is a snapshot of the main window of PROBE, displayed by thePDA component for the basic input data in Table 2. The portfolioselected by the optimization approach, solving the knapsackproblem (8) for a budget of 5 million, is the efcient portfolio (4.8,414.25),1 indicated by a star dot in the graph, with a cost of 4.8million, a benet of 414.25, and composed of ve projects, P03 andP04 from dept. A, P09 and P10 from dept. C, and P12 from dept. D,as highlighted in the tables above the graph.

This optimal portfolio (4.8, 414.25) is convex efcient; therefore, it isalso the one selected by the prioritization approach, as shown in Table 3.

The director of dept. B argues against the selection of portfolio (4.8,414.25) because it does not include any project from his department.The DMG decides to analyze the potential loss of benet associated toimposing that at least one project from each department be selected ifpossible. Accordingly, four constraints of type jGxjq with q=1are added to PROBE, giving rise to the new results shown in Fig. 8.

B1 B2 B3 B4

P01 A 1.1 67 40 30 50 47.57P02 A 1.9 55 37 40 20 39.88P03 A 0.9 100 90 80 90 90.20P04 A 0.9 90 80 70 95 83.35P05 B 1.8 48 35 40 33 40.06P06 B 1.3 43 32 25 44 35.90P07 B 1.3 42 64 80 55 59.93P08 B 1.1 40 50 44 20 38.86P09 C 0.9 80 90 78 66 78.38P10 C 1.3 88 66 80 70 77.72P11 D 0.7 41 48 49 25 41.29P12 D 0.8 86 88 90 72 84.60

=14Weights 0.31 0.19 0.29 0.21The optimal portfolio is now (4.8, 396.46), which coincidentallycosts the same as portfolio (4.8, 414.25) but offers 17.79 less unitsof benet: the result of having replaced P10 with P07, as well-notedby the director of dept. C.

The robustness of selecting portfolio (4.8, 396.46) is then evaluatedfor an uncertainty domain dened by all benet value scores of allprojects on all criteria varying10 units and all weights varyingwithinthe bounds indicated in Table 4 simultaneously, subjected to a normal-ization constraint.

Fig. 9 is a snapshot of the robustness analysis window of the PDAcomponent of PROBE, showing that the only competitor of the pro-posed portfolio (4.8, 396.46) is portfolio (4.6, 375.39), because it isthe only restricted efcient portfolio relative to the proposed portfo-lio. The top-right table in Fig. 9 shows that the two portfolios onlydiffer on one project, with project P07 of dept. B, included in the pro-posed portfolio, being replaced in the competitor portfolio by projectP08, also of dept. B.

1 For the sake of brevity, we will use the ordered pair (x, y) to denote the portfoliowith cost x million and benet y.Some doubts are raised about the cost of project P08 being smallerthan the cost of project P07. The cost of P08 is then allowed to varywithin the range [1.1 million, 1.5 million] and this uncertainty isadded to the previous uncertainty domain. The top-left table inFig. 10 shows that there is no new competitor portfolio and the onlylimit that changes is, obviously, the maximum difference in costbetween the proposed and the competitor portfolios (MaxDifCost).

The ranges of variation of the differences in cost and benet betweenthese two portfolios are plotted in the top-right graph in Fig. 10. Thereare two areas shaded in this graph: the proposed portfolio is betterboth in cost and benet in the robustness area at the right (displayedin red in PROBE) where those two differences are positive, whereas thecompetitor portfolio is better in the regret area at the left (displayed inblue in PROBE) where the two differences are negative.

6. Case study

In this section, we describe briey an application of PROBE tohealth service planning in Northern Lisbon [52,57], which demon-strates the usefulness of the software and the approach. It is widelyrecognized that multicriteria portfolio approaches have considerablepotential in the planning of health services [1]. This is partly becausethe benets of investment in healthcare are inherently multi-dimensional and difcult to assess and tradeoff, but also becausedelivery requires the participation of members of multiple clinicalspecialties, and so approaches to decision aiding which facilitatestakeholder engagement are particularly appropriate.

The client in this case study was the Clinical Board of the Group ofHealth Centers (GHC) of Northern Lisbon, a collective organizationcomprising Health Centers for the districts of Lumiar, Sete Rios,Benca and Alvalade. The Board was faced with the task of planninga new Community Care Division (CCD) within each Health Center.Analysis of the action plan for each CCD showed that the Health Centerswere not adequately staffed to deliver all the community care projectswhich they wished to. Hence a decision had to be taken about whichprojects to undertake.

In order to support the client, a multicriteria portfolio decision anal-ysis was conducted through a series of working modeling meetings ordecision conferences [56] with the Clinical Board and the ExecutiveDirector of the GHC. Four criteria (effective health gains, equity, achieve-ment of GHC goals, and t with existing services and community needs)were dened and project benet was assessed against these criteriausing the MACBETH approach [3,5] (detailed descriptions of MACBETHapplications can be found, e.g., in [5,8,10]).

Here we focus on the Sete Rios action plan, which was composedof 14 possible projects. The core constraint was the shortfall in avail-able nursing hours: to do all projects would require 18,450 nursinghours, but only 17,460 nursing hours were available, and hiringmore nurses was not possible. Additionally, ve other constraintswere also dened: one of the projects, the situation diagnosis project,should be included in the selected portfolio and the hours available ofother health service professionals should not be exceeded, namely,1665 social service ofcer hours, 990 psychologist hours, 810 oralhygienist hours and 570 general practitioner hours. The costbenetplot for Sete Rios under this scenario is shown in Fig. 11. The gap inthe display between the bottom left and top right clusters of pointsis due to the existence of a very large project, integrated long-termcare (which, after discussion, the Board decided to implement).

The most attractive portfolio (the proposed portfolio in PROBEterminology) costs 15,300 nursing hours and gives 636.86 benetunits (it is marked with a star dot on the graph in Fig. 11). Althoughthe proposed portfolio is not using the available nursing hours asmuch as possible (2160 nursing hours still are available), the ClinicalBoard and the Executive Director of the GHC did not want to selectmore costly portfolios because they were felt to add little benet

for their added expenditure in nursing hours.

544 J.C. Loureno et al. / Decision Support Systems 54 (2012) 534550The robustness of the proposed portfolio was tested under condi-tions of uncertainty on the part of the Clinical Board, modeled byintervals on the criteria weights, which were allowed to vary withinthese intervals. Also, the benet value scores on effective healthgains and achievement of GHC goals were allowed to vary withinranges to reect both individual uncertainty and interpersonal dis-agreement between the Clinical Board and the team from the CCD.

Table 3Projects ranked by decreasing benet-to-cost ratio (order of priority).

Project Benet Cost Ratio Cumulative cost

P12 84.60 0.8 105.8 0.8P03 90.20 0.9 100.2 1.7P04 83.35 0.9 92.6 2.6P09 78.38 0.9 87.1 3.5P10 77.72 1.3 59.8 4.8P11 41.29 0.7 59.0 5.5P07 59.93 1.3 46.1 6.8P01 47.57 1.1 43.2 7.9P08 38.86 1.1 35.3 9.0P06 35.90 1.3 27.6 10.3P05 40.06 1.8 22.3 12.1P02 39.88 1.9 21.0 14.0

Fig. 7. PROBE analysis of efcient portfolios. Each row of the top-left table shows informationconvex efcient or non-convex efcient, respectively; a 1 or a 0 in each one of the columns Pportfolio. The top-right table window shows the projects structured within areas (the four dethe star dot indicates the selected portfolio), with the convex efcient ones linked by a dotThe PROBE robustness analysis showed that given uncertainty aboutcriterion weights and project benet there were 28 competitor port-folios. It emerged in the course of analysis and discussion within thegroup that none of these competitor portfolios included the projectintegrated long-term care. The group agreed that it was not worth-while losing this project, and so the proposed portfolio could beconsidered a robust choice. We note that unused nursing hours arenotwasted: unused nurses can be released to other tasks not consideredin this analysis or might be assigned to reformulated versions ofnon-selected projects.

The Clinical Board seems to have found that the analysis was usefulto them in arriving at a decision about what to do, and also that it chal-lenged them to develop their own information system to provide moredecision-relevant information. Moreover, the case study has receivedbroad recognition in the professional community, being awarded thebest paper prize at the 12th Conference of the Portuguese Associationof Health Economics.

7. Conclusion

Although the PROBE decision support system has many features incommon with the commercial portfolio decision analysis software

about one efcient portfolio: a 1 or a 0 in column CE indicates whether the portfolio is01 to P12 indicates whether the respective project is included or not, respectively, in thepartments). Each efcient portfolio is represented by a dot in the bottom graph (whereted line.

545J.C. Loureno et al. / Decision Support Systems 54 (2012) 534550analyzed in [46], it goes a step forward because it identies all convexefcient and non-convex efcient portfolios and enables users toperform in-depth robustness checks on proposed portfolios.

In the RPM approach, referred to in Section 1, the decision-maker isadvised to start with loose preference statements, which imply largefeasible sets of the parameter values and typically result in a large num-ber of non-dominated portfolios [43, p. 683]. The decision-maker isthen invited to narrow the initial uncertainty domain, therefore reduc-ing the number of non-dominated portfolios, and decision rules can beconsulted to recommend one of the remaining non-dominated portfoli-os [43, p. 683]. In contrast, PROBE supposes the decision-maker can usebest-guess parameter values to nd an attractive proposed portfolio,and then offers a form of robustness analysis in which the proposedportfolio is compared to competitors in its neighborhood.

Fig. 8. PROBE analysis of efcient po

Table 4Uncertainty on the weights.

Criterion Weight Lower bound Upper bound

B1 0.31 0.26 0.36B2 0.19 0.14 0.24B3 0.29 0.24 0.34B4 0.21 0.16 0.26Despite the simplicity of the illustrative example presented inSection 5, PROBE can deal with complex portfolio decision analysisproblems that involve a signicant number of projects (see AppendixA) and many interactions among them. When the user denes a largeuncertainty domain (giving rise to a signicant number of competitorportfolios, and therefore rendering inconclusive the analysis of robust-ness of the portfolio choice) the user may be invited, as in RPM, toprogressively reduce the uncertainty domain, a reasoning called pro-gressive reduction of incomparability in [11, Section 3.2] and alreadypresent in [58, p. 258]. The identication of the projects common tothe proposed and the restricted efcient portfoliosthe core projectsof PROBE and RPMis in our view one of themost important outputs ofa portfolio robustness analysis.

It is usually advantageous to rst run PROBE without adding con-straints to the simple model (8); indeed, this will permit highlightingthe best strategic portfolio choices [30,56]. Subsequently, structural, op-erational, programmatic or planning constraints [30,37,53] can beadded interactively, as necessary, to make the model more realisticand satisfy the requirements [54] for selecting a robust portfolio. Ourexperience in using the EQUITY software to support portfolio decisionanalysis [4,6,7,56] has revealed that although several types of con-straints can be built in manually and visually [15,56] into an EQUITYmodel, this is not always an easy task and occasionally not even possi-ble. Consequently, although portfolio robustness evaluation has been

rtfolios with group constraints.

Fig. 9. First portfolio robustness evaluation. Each row of the top-left table shows information about one competitor portfolio (i.e., a restricted efcient portfolio relative to the pro-posed portfolio): the four leftmost columns show the minimum and the maximum differences in cost and in benet between the proposed portfolio and each competitor portfolio;a 1 or a 0 in each one of the columns P01 to P12 indicates whether the respective project is included or not, respectively, in the competitor portfolio. The top-right table windowshows the projects included in the proposed portfolio; the projects included in the competitor portfolio highlighted in the table at the left and indicated by a surrounding circle in thegraph; the projects common to both portfolios; and the projects that are not common to both portfolios and their respective areas (departments in our example).

Fig. 10. Second portfolio robustness evaluation.

546 J.C. Loureno et al. / Decision Support Systems 54 (2012) 534550

vs

C

t g

547J.C. Loureno et al. / Decision Support Systems 54 (2012) 534550the core motivation for the conception of PROBE, its ability to formallyaccommodate project interactions and other constraints has been animportant complementary software design objective.

We anticipate continuing to develop PROBE and to do so will raiseinteresting methodological and conceptual challenges. For example,in the version of PROBE reported in this paper, we do not deal withthe question of how to analyze projects where there are benets asso-ciated with not doing a project (a discussion on this topic is presentedin [18]). One can envisage different ways of modeling such asituation; some approaches (e.g. replacing the objective function

in Eq. (8) with maxmj1 vjxj v0j 1xj h i

where vj0 is the benet

value of not selecting project j [18,35]) might require modicationof the PROBE software and algorithms. We leave the question ofhow best to model and handle this case for further study, however.

The importance of carefully structuring the resource allocationproblem [49], wisely assessing the benet values of the projects,namely to escape from the semi-global scaling effect [51, p. 270],and correctly weighting the benet criteria, should also be emphasized

Cost

8,0006,0004,0002,000

Ben

efit

700650600550500450400350300250200150100500

Fig. 11. Costbeneas essential prerequisites for a successful multicriteria portfolio analysisbased on additive value measurement [8,34]. In particular, it is funda-mental to avoid the most common critical mistake [32, p. 147],which is unfortunately made by many popular weighting proceduresthat assignweights to criteria simply on the basis of the intuitive notionof importance; this is why PROBE includes a module that guides theuser through a step-by-step procedure to swing weighting the criteria[23]. In addition, we should prevent the use of outputs from an additivevalue model as benet inputs for the portfolio analysis without rstdening the meaning of the zero benet value for a portfolio (see[7,18,30]). Last but not least, we emphasize that the technical compo-nent provided by the decision support system PROBE cannot, by itself,avoid all of the aforementioned traps; this is why facilitators ofsocial-technical multicriteria portfolio selection processes [56] shouldbe not only experienced in working with groups but also skillful indecision analysis technical principles.

Acknowledgments

The authors thank Ana Respcio, Joo Oliveira Soares, Manuel PedroChagas, Mnica Duarte Oliveira, Teresa Cipriano Rodrigues and partici-pants in theNewYorkmeeting of the International Decision ConferencingForum for their helpful comments. The authors acknowledge the refereesfor their careful review and useful comments. The authorswould also liketo thank the members of the Clinical Board of the North Lisbon Group ofHealth Centers that participated in the development and application ofthe model of the case study, briey described in this paper. The authorsgratefully acknowledge the support of the RAMS grant from the Councilof Rectors of Portuguese Universities and the British Council under theTreaty of Windsor program. Joo C. Loureno and Carlos A. Bana e Costagratefully acknowledge the support of the FCT (Portuguese National Sci-ence Foundation) under project PTDC/GES/73853/2006.

Appendix A

Specications

The running times presented in this section were obtained whileusing a single thread in a computer with an Intel Core 2 Duo T78002.60 GHz CPU, 4 GB of RAM and Windows 7 Professional 64 bit.The MILP solver lp_solve 5.5.2.0 was used by the algorithmsFindEfcientPortfolios, FindCandidates and FindRestEfcientPortfolios.

Benefit

ost18,00016,00014,00012,00010,000

raph for Sete Rios.The input data used in the trials were generated in a Microsoft Excelworksheet using the function RANDBETWEEN(a,b), which returnsan uniformly distributed pseudo-random integer number betweenthe lower bound a and the upper bound b specied by the user. Thecosts of the projects were integers generated between 1 and 10 andthe benet value scores of the projects on four benet criteria wereintegers generated between 1 and 100. The data used in the trialswere generated for 100 projects, with no interactions among them.The projects were divided into three sets: Set no. 1includes therst 50 projects listed on the worksheet; Set no. 2includes the rst75 projects listed; Set no. 3includes the 100 projects listed. Theweights of three benet criteria (w1,w2 and w3) were integers gener-ated between 1 and 30 and divided by 100, which resulted in w1=0.19, w2=0.30 and w3=0.27, and the weight of the fourth criterionwas calculated as w4=1(w1+w2+w3)=0.24.

Computation of the efcient frontier

Table 5 shows the running times of the PROBE algorithmFindEfcientPortfolios, which was used to nd the complete efcientfrontier, including possible multiple optimal portfolios, for each ofthe three sets of projects. The in algorithm FindEfcientPortfolios isautomatically set by PROBE based on the number of signicant guresto which the costs are expressed: in this case since the costs of the

projects are integer numbers between 1 and 10 the minimum differ-ence in cost that may occur between portfolios is 1, and hence the was automatically set by PROBE to 1. The running times of the algo-rithm FindConvexEfcientPortfolios are not referred to in Table 5, be-cause it took less than one second to nd all the convex efcientportfolios in the three trials.

Robustness evaluation

Table 6 shows the running times of the PROBE algorithmsFindCandidates and FindRestEfcientPortfolios for the three sets ofprojects used in the trials. Uncertainties regarding the benet valuescores of the projects, the costs of the projects and the criteriaweights were dened as follows. The uncertainty on the benetvalue vij of each project j on each benet criterion i was dened asan interval such that vijvvijvij+v (with 1vij100), for alli and j; the uncertainty on the cost cj of each project j was dened

portfolios of Set no. 1, Set no. 2 and Set no. 3, respectively, with thefollowing costs and benets: (89, 1467.27); (132, 2356.11); (181,3016.75).

Final notes

Due to the combinatorial nature of the knapsack problem the run-ning times of the PROBE algorithms increase greatly with a small in-crement in the number of projects, e.g. in our trials it took 11 s tond the efcient frontier for 100 projects whereas for 50 projects itonly took 1 s (see Table 5). The in algorithm FindEfcientPortfoliosmay also have a signicant impact in the running times, e.g. if wehad set =0.1 in the trial with the 100 projects reported in Table 5PROBE would have taken 1 min and 38 s to nd the entire efcientfrontier instead of the 11 s it took with =1; thus, should be setto a number small enough to allow nding all efcient portfoliosbut not lower than that. The robustness evaluation is (obviously) af-fected by the number of projects in the trials, by the uncertaintyranges of its parameters (wider uncertainties imply increased run-ning times), and by the simultaneous uncertainty on several parame-ters of the model; e.g. in the trial with Set no. 3 the algorithmFindCandidates took 30 s to nd all candidate portfolios when consid-ering simultaneous variations on the benet value scores of the pro-jects on the four criteria of v=2.5 and on the costs of all projectsof c=0.025, whereas when variations on the weights of the fourbenet criteria of w=0.025 were added, the algorithm took 3 minand 37 s to complete (see Table 6) a more dramatic increase inthe running time was found when c was increased to 3 (the algo-rithm FindCandidates took 1 h 20 min and 42 s to complete). In con-

Table 5Running times of algorithm FindEfcientPortfolios.

Setno.

Numberofprojects

Sum of thecosts of allprojects

Sum of thebenets of allprojects

Number ofefcientportfolios found

Runningtime

1 50 278 2587.59 238 1 s2 75 399 3945.16 366 4 s3 100 555 5143.37 536 11 s

or)

0 0 0.025 0 b1 s

548 J.C. Loureno et al. / Decision Support Systems 54 (2012) 5345500.025 2.5 0 6 b1 s0.025 0 0.025 1 b1 s0 2.5 0.025 1 b1 s0.025 2.5 0.025 9 1 s0.025 3 0.025 16 2 s

2 0.025 0 0 1 b1 s0 2.5 0 1 b1 s0 0 0.025 0 b1 sas an interval such that (1c) cjcj(1+c) cj, for all j; and the un-certainty on the benet criterion weight wi was dened as an intervalsuch that (1w) wiwi(1+w) wi, for all i. For each of the threesets of projects used in the trials, the robustness of the convex efcientportfolio was evaluated with the cost near to (but not higher than) onethird of the sum of the costs of all projects in that set (see Table 5). Therobustness evaluations were performed for the convex efcient

Table 6PROBE running times for robustness evaluation.

Set no. Uncertaintyparameters

No. of candidateportfolios found

Time spent searching fcandidate portfolios (a

w v c

1 0.025 0 0 1 b1 s0 2.5 0 1 b1 s0.025 2.5 0 8 b1 s0.025 0 0.025 1 b1 s0 2.5 0.025 1 b1 s0.025 2.5 0.025 13 3 s0.025 3 0.025 42 22 s

3 0.025 0 0 0 b1 s0 2.5 0 2 b1 s0 0 0.025 0 b1 s0.025 2.5 0 5 b1 s0.025 0 0.025 0 b1 s0 2.5 0.025 46 30s0.025 2.5 0.025 126 3 m 57 s0.025 3 0.025 717 1 h 20 m 42 s

Note. Running times for the algorithms (a) FindCandidates and (b) FindRestEfcientPortfoliostrast, these changes in parameters have no noticeable impact on therunning times of the algorithm FindRestEfcientPortfolios (see Table 6).

In order to reduce the PROBE running times or to address more com-plex problems (i.e. problems with more projects and/or with wideruncertainty domains) and taking advantage of multiple core computers,we could run several instances of the PROBE algorithms in parallelusing multiple threads (namely the most time consuming algorithms,

No. of restricted efcientportfolios found

Time spent searching forrestricted efcient portfolios (b)

Total runningtime

1 b1 s b1 s1 b1 s b1 s0 0 s b1 s3 b1 s b1 s1 b1 s b1 s1 b1 s b1 s3 b1 s 1 s8 b1 s 2 s1 b1 s b1 s1 b1 s b1 s0 0 s b1 s4 b1 s b1 s1 b1 s b1 s1 b1 s b1 s4 b1 s 3 s

13 b1 s 22 s0 0 s b1 s2 b1 s b1 s0 0 s b1 s4 b1 s b1 s0 0 s b1 s

46 b1 s 30s110 b1 s 3 m 57 s585 b1 s 1 h 20 m 43 s.

549J.C. Loureno et al. / Decision Support Systems 54 (2012) 534550FindEfcientPortfolios and FindCandidates, although that would requiremaking some adjustments on the algorithms herein presented), and/orwe could replace the lp_solve solver by an industrial strength solv-er capable of generating multiple threads (e.g. the CPLEX solver,http://www-01.ibm.com/software/integration/optimization/cplex-optimizer/). However, as the free lp_solve solver delivers accept-able solution times for realistic problems, we have retained the for-mer option for PROBE.

References

[1] M. Airoldi, A. Morton, Portfolio decision analysis for population health, in: A. Salo,J. Keisler, A. Morton (Eds.), Portfolio Decision Analysis: Improved Methods forResource Allocation, Springer, New York, 2011, pp. 359381.

[2] C.A. Bana e Costa, The use of multi-criteria decision analysis to support the searchfor less conicting policy options in a multi-actor context: Case study, J.Multi-Criteria Decis. Anal. 10 (2001) 111125.

[3] C.A. Bana e Costa, M.P. Chagas, A career choice problem: An example of how touse MACBETH to build a quantitative value model based on qualitative valuejudgments, Eur. J. Oper. Res. 153 (2004) 323331.

[4] C.A. Bana e Costa, M.L. Costa-Lobo, I.A.J. Ramos, J.C. Vansnick, Multicriteria ap-proach for strategic town planning: The case of Barcelos, in: D. Bouyssou, E.Jacquet-Lagrze, P. Perny, R. Slowinsky, D. Vanderpooten, P. Vincke (Eds.), AidingDecisions with Multiple Criteria: Essays in Honour of Bernard Roy, Kluwer Aca-demic Publishers, Dordrecht, 2002, pp. 429456.

[5] C.A. Bana e Costa, J.M. De Corte, J.C. Vansnick, MACBETH, Int. J. Inf. Technol. Decis.Mak. 11 (2012) 359387.

[6] C.A. Bana e Costa, L. Ensslin, E.C. Corra, J.C. Vansnick, Decision support systems inaction: Integrated application in a multicriteria decision aid process, Eur. J. Oper.Res. 113 (1999) 315335.

[7] C.A. Bana e Costa, T.G. Fernandes, P.V.D. Correia, Prioritisation of publicinvestments in social infrastructures using multicriteria value analysis anddecision conferencing: A case study, Int. Trans. Oper. Res. 13 (4) (2006)279297.

[8] C.A. Bana e Costa, J.C. Loureno, M.P. Chagas, J.C. Bana e Costa, Development of re-usable bid evaluation models for the Portuguese Electric Transmission Company,Decis. Anal. 5 (2008) 2242.

[9] C.A. Bana e Costa, J.C. Loureno, J.O. Soares, An interval weighting assignmentmodel for credit analysis, J. Financ. Decis. Mak. 3 (2007) 19.

[10] C.A. Bana e Costa, C.S. Oliveira, V. Vieira, Prioritization of bridges and tunnels inearthquake risk mitigation using multicriteria decision analysis: Application toLisbon, Omega 36 (2008) 442450.

[11] C.A. Bana e Costa, P. Vincke, Measuring credibility of compensatory preferencestatements when trade-offs are interval determined, Theory Decis. 39 (1995)127155.

[12] D. Bouyssou, T. Marchant, M. Pirlot, A. Tsoukis, P. Vincke, Evaluation and Deci-sion Models with Multiple Criteria: Stepping Stones for the Analyst, Springer,New York, 2006.

[13] R.A. Brealey, S.C. Myers, Principles of Corporate Finance, 7th ed. McGraw-Hill/Irwin,Boston, 2003.

[14] G.G. Brown, R.F. Dell, A.M. Newman, Optimizing military capital planning, Inter-faces 34 (2004) 415425.

[15] D. Buede, T. Bresnick, Decision analysis for resource allocation decisions, IDI Tech-nical Report, Innovative Decisions, Inc., Vienna, VA, 2000.

[16] D.M. Buede, T.A. Bresnick, Applications of decision analysis to the military sys-tems acquisition process, in: W. Edwards, R.F. Miles Jr., D. von Winterfeldt(Eds.), Advances in Decision Analysis: From Foundations to Applications,Cambridge University Press, Cambridge, 2007, pp. 539563.

[17] Catalyze Ltd., Equity (version 3.4), http://www.catalyze.co.uk, 2008.[18] R.T. Clemen, J.E. Smith, On the choice of baselines in multiattribute portfolio anal-

ysis: A cautionary note, Decis. Anal. 6 (2009) 256262.[19] R.G. Cooper, S.J. Edgett, E.J. Kleinschmidt, New product portfolio management:

Practices and performance, J. Prod. Innov. Manage. 16 (1999) 333350.[20] R.G. Cooper, S.J. Edgett, E.J. Kleinschmidt, Portfolio Management for New Prod-

ucts, 2nd ed. Perseus Books, Cambridge, MA, 2001.[21] G.B. Dantzig, Discrete-variable extremum problems, Oper. Res. 5 (1957) 266277.[22] W. Edwards, How to use multiattribute utility measurement for social

decisionmaking, IEEE Trans. Syst. Man Cybern. 7 (1977) 326340.[23] W. Edwards, F.H. Barron, SMARTS and SMARTER: Improved simple methods for

multiattribute utility measurement, Organ. Behav. Hum. Decis. Process. 60(1994) 306325.

[24] M. Ehrgott, Multicriteria Optimization, 2nd ed. Springer, Berlin, 2005.[25] Expert Choice, Expert Choice (version 11.4 with the Resource Aligner extension),

http://www.expertchoice.com, 2007.[26] M.R. Garey, D.S. Johnson, Computers and Intractability: A Guide to the Theory of

NP-Completeness, Freeman, San Francisco, 1979.[27] F. Ghasemzadeh, N.P. Archer, Project portfolio selection through decision support,

Decis. Support Syst. 29 (2000) 7388.[28] K. Golabi, Selecting a portfolio of nonhomogeneous R&D proposals, Eur. J. Oper.

Res. 21 (1985) 347357.[29] K. Golabi, Selecting a group of dissimilar projects for funding, IEEE Trans. Eng.

Manage. 34 (1987) 138145.[30] K. Golabi, C.W. Kirkwood, A. Sicherman, Selecting a portfolio of solar energyprojects using multiattribute preference theory, Manage. Sci. 27 (1981)174189.

[31] P. Gunther, Use of linear programming in capital budgeting, J. Oper. Res. Soc. Am.3 (1955) 219224.

[32] R.L. Keeney, Value-Focused Thinking: A Path to Creative Decisionmaking, HarvardUniversity Press, Cambridge, MA, 1992.

[33] H. Kellerer, U. Pferschy, D. Pisinger, Knapsack Problems, Springer, Berlin, 2004.[34] C.W. Kirkwood, Strategic Decision Making: Multiobjective Decision Analysis with

Spreadsheets, Duxbury Press, Belmont, CA, 1997.[35] C.W. Kirkwood, How to take account of the value of not selecting an alternative,

http://www.public.asu.edu/~kirkwood/SDMBook/Not_selecting_an_alternative.pdf, 2009.

[36] C.E. Kleinmuntz, D.N. Kleinmuntz, A strategic approach to allocating capitalin healthcare organizations, Healthc. Financ. Manage. 53 (1999) 5258.

[37] D.N. Kleinmuntz, Resource allocation decisions, in: W. Edwards, R.F. Miles Jr., D.von Winterfeldt (Eds.), Advances in Decision Analysis: From Foundations to Ap-plications, Cambridge University Press, Cambridge, 2007, pp. 400418.

[38] A. Ko, D.P. Morton, E. Popova, S.M. Hess, E. Kee, D. Richards, Prioritizing projectselection, Eng. Econ. 54 (2009) 267297.

[39] B. Korte, J. Vygen, Combinatorial Optimization: Theory and Algorithms, 3rd ed.Springer, Berlin, 2006.

[40] Krysalis, HiPriority (version 3), http://www.krysalis.co.uk, 2007.[41] J. Liesi, Portfolio Decision Analysis for Robust Project Selection and

Resource Allocation, PhD Thesis, Department of Engineering Physics andMathematics, Systems Analysis Laboratory, Helsinki University of Technology,Espoo, 2008.

[42] J. Liesi, P. Mild, A. Salo, Preference programming for robust portfolio modelingand project selection, Eur. J. Oper. Res. 181 (2007) 14881505.

[43] J. Liesi, P. Mild, A. Salo, Robust portfolio modeling with incomplete cost informa-tion and project interdependencies, Eur. J. Oper. Res. 190 (2008) 679695.

[44] Logical Decisions, Logical Decisions Portfolio (with Logical Decisions, version 6.1),http://www.logicaldecisions.com, 2008.

[45] J.H. Lorie, L.J. Savage, Three problems in rationing capital, J. Bus. 28 (1955)229239.

[46] J.C. Loureno, C.A. Bana e Costa, A. Morton, Software packages for multi-criteriaresource allocation, Proceedings of the IEEE International Engineering Manage-ment Conference, IEMC-Europe 2008, Estoril, Portugal, 2008.

[47] S. Martello, P. Toth, The 01 knapsack problem, in: N. Christodes, A. Mingozzi, P.Toth, C. Sandi (Eds.), Combinatorial Optimization, John Wiley & Sons, Chichester,1979, pp. 237279.

[48] S.Martello, P. Toth, Knapsack Problems: Algorithms and Computer Implementations,John Wiley & Sons, Chichester, 1990.

[49] G. Montibeller, L.A. Franco, E. Lord, A. Iglesias, Structuring resource allocationdecisions: A framework for building multi-criteria portfolio models with area-grouped options, Eur. J. Oper. Res. 199 (2009) 846856.

[50] A. Morton, On the choice of baselines in portfolio decision analysis, WorkingPaper LSEOR 10.128, London School of Economics, London, 2010. http://www.lse.ac.uk/collections/operationalResearch/research/workingPapers.htm.

[51] A. Morton, B. Fasolo, Behavioural decision theory for multi-criteria decision anal-ysis: A guided tour, J. Oper. Res. Soc. 60 (2009) 268275.

[52] M.D. Oliveira, T.C. Rodrigues, C.A. Bana e Costa, A. Brito de S, Prioritizing health careinterventions: Amulticriteria resource allocationmodel to inform the choice of com-munity care programmes, in: E. Tnfani, A. Testi (Eds.), Advanced Decision MakingMethods Applied to Health Care, Springer, Milan, 2012, pp. 141154.

[53] G.S. Parnell, G.E. Bennett, J.A. Engelbrecht, R. Szafranski, Improving resourceallocation within the National Reconnaissance Ofce, Interfaces 32 (2002)7790.

[54] L.D. Phillips, A theory of requisite decision models, Acta Psychol. 56 (1984) 2948.[55] L.D. Phillips, Decision conferencing, in: W. Edwards, R.F. Miles Jr., D. von

Winterfeldt (Eds.), Advances in Decision Analysis: From Foundations to Applica-tions, Cambridge University Press, Cambridge, 2007, pp. 375399.

[56] L.D. Phillips, C.A. Bana, e Costa, Transparent prioritisation, budgeting and resourceallocation with multi-criteria decision analysis and decision conferencing, Ann.Oper. Res. 154 (2007) 5168.

[57] T.C. Rodrigues, Modelo Multicritrio de Afectao de Recursos Humanos emProjectos de Cuidados Comunidade, MSc Thesis, Instituto Superior Tcnico -Universidade Tcnica de Lisboa, Faculdade de Medicina da Universidade deLisboa, Lisboa, 2010.

[58] B. Roy, Problems and methods with multiple objective functions, Math. Program.1 (1971) 239266.

[59] B. Roy, A missing link in OR-DA: Robustness analysis, Found. Comput. Decis. Sci.23 (1998) 141160.

[60] A. Salo, J. Keisler, A. Morton (Eds.), Portfolio Decision Analysis: ImprovedMethods for Resource Allocation, Springer, New York, 2011.

[61] A. Salo, P. Mild, T. Pentikinen, Exploring causal relationships in an innovationprogram with robust portfolio modeling, Technol. Forecast. Soc. Change 73(2006) 10281044.

[62] P. Sharpe, T. Keelin, How Smithkline Beecham makes better resource-allocationdecisions, Harv. Bus. Rev. 76 (1998) 4557.

[63] B. Stroustrup, The C++ Programming Language, 3rd ed. Addison-Wesley, Read-ing, MA, 1997.

[64] H.M. Weingartner, The excess present value indexA theoretical basis andcritique, J. Account. Res. 1 (1963) 213224.

[65] M. Weinstein, R. Zeckhauser, Critical ratios and efcient allocation, J. Public Econ.2 (1973) 147157.

Joo C. Loureno is Assistant Professor of decision analy-sis, operational research, and production and operationsmanagement at the Technical University of Lisbon, Schoolof Engineering (IST), Department of Engineering and Man-agement. Before joining IST, he worked for seven years as adecision analyst for a consultancy company. He graduatedin Applied Mathematics, he holds an MSc in OperationalResearch and Systems Engineering, and a PhD in IndustrialEngineering and Management. He is a member of the Cen-ter for Management Studies of IST. His research has beenpublished in Decision Analysis, the European Journal ofOperational Research, and the Journal of Financial DecisionMaking. He has been involved in consulting projects inPortugal and Brazil, and also for the European Commission.

Alec Morton is Senior Lecturer in Management Science inthe Department of Management at the London School ofEconomics and Political Science. He teaches courses in deci-sion analysis, simulation, and statistics, and his researchinterests are in the application of decision analysis to plan-ning problems, especially in healthcare; in Multicriteria De-cision Analysis and Multiobjective Optimization; in thenormative foundations of health economics, and in games ofattack and defence.With Ahti Salo and Jeff Keisler, he is an ed-itor of "Portfolio Decision Analysis: Improved Methods for Re-source Allocation" published in Springer's International Seriesin Operations Research &Management Science. He is a gradu-ate of the Universities of Manchester and Strathclyde, and be-fore joining the LSE, worked at Singapore Airlines and theNational University of Singapore.

Carlos A. Bana e Costa is Full Professor of Decision andInformation at the Technical University of Lisbon, School of En-gineering (IST), Department of Engineering and Management,and he was Visiting Professor of Decision Sciences at theLondon School of Economics and Political Science, Departmentof Management (1999-2010). He is also head of research pro-jects at the Center of Management Studies of IST. His primaryinterests have been in the elds of Management and DecisionSciences, namely Multicriteria Decision Analysis and DecisionConferencing. He has published widely in these areas and heis co-author of the MACBETH approach for decision-aiding(http://www.m-macbeth.com). He has also been developingconsultation in public strategic decision-making processes,policy appraisal, andbid and suppliersperformance evaluation

throughout the world, following the socio-technical facilitation perspective shared by themembers of the International Decision Conferencing Forum. He is also a senior partner ofBANA Consulting (http://www.bana-consulting.pt).

550 J.C. Loureno et al. / Decision Support Systems 54 (2012) 534550

PROBEA multicriteria decision support system for portfolio robustness evaluation1. Introduction2. Basic concepts and portfolio selection approaches3. Introducing the DSS PROBE3.1. The MCDA and PDA components and basic input data3.2. Inputting data for modeling project interactions3.2.1. Synergies among projects3.2.2. Constraints on projects

4. PROBE innovative functionalities4.1. Finding all efficient portfolios within a given portfolio cost range4.2. Portfolio robustness evaluation

5. Example6. Case study7. ConclusionAcknowledgmentsAppendix ASpecificationsComputation of the efficient frontierRobustness evaluationFinal notes

References