CHAPTER 20

Social Choice on the Web

Sylvain Bouveret

In this chapter, we will speak about the development and use of a web applicationdedicated to social choice, Whale.1 The chapter is intended to be a compilationof lessons learned from the users of this web application, rather an extensivepresentation of it. We will see through specific use cases what concrete problemsthat have been encountered during the development of this web application.

20.1 Introduction

The history of social choice theory is paved with paradoxes. Of course, theoreticalpeculiarities like the Condorcet Paradox or Arrow’s Theorem form the basis ofthe discipline. But social choice even seems to be inherently paradoxical at theepistemological level.

First example: Social Choice Theory has been being studied in its modernform for two centuries — if we date back the birth of modern social choice tothe debates between Condorcet and Borda’s in the late 18th century. During thehistory of this discipline, a lot of theoretical knowledge about collective decisionmaking and voting procedures has been accumulated. Yet, until recently, theoccasions to really put this knowledge into practice in everyday life were ratherinfrequent and limited to political elections or particular situations (like scientificcommittees). As a consequence of this distortion between theory and practiceof social choice, lessons and knowledge learned from the theoretical study ofcollective decision making did not seem to have much influence on the way peoplepractice elections.2 This can be well explained. In the rare occasions wherepeople really practice voting, the stakes and the exogenous influences (political,cultural,...) are usually so high that nobody would ever think of questioning thevoting process itself.

Second example: although people only seldom experiment with voting, practi-cal situations of making collective decisions happen every day. A group of friendsdeciding which movie to watch, colleagues choosing a date for a meeting, a re-cruitment committee hiring a candidate for a job,... All these situations that

1WHich ALternative is Elected. Current version accessible at: https://whale.imag.fr/2The converse can also be observed to some extent, where the models at the heart of social choice

are often based on very strong assumptions, the practical relevance of which can be questioned, orat least need to be confronted with experiment.

2 S. Bouveret

usually do not require a formal vote are nevertheless collective decision makingsituations. These everyday life situations would be perfect candidates for apply-ing the formal knowledge brought by voting theorists. Usually the stakes remainquite low, but are still high enough to require an enlightened decision that votingtheory can provide.

Bringing voting theory to these kinds of everyday situations was still hardlyconceivable until recently. One major reason is that the logistical burden re-quired to implement the least voting procedure (expressing a linear order inpractice, tallying the votes,...) is often too high to be reasonable. Another majorreason is that most people are simply not aware of the fact that it is possible tovote other than by simply writing one name on the ballot sheet and choosing asthe winner the candidate voted for most often.3

Two recent major scientific and technological breakthroughs that have noth-ing to do with social choice have dramatically changed things. The first one isthe advent of computer networks and the World Wide Web. This has a major im-pact on the democratisation of social choice for several reasons. First, it makesthe implementation of light remote voting systems possible. Human computerinterfaces make possible the use of ways of expressing preferences that wouldhave been unpractical otherwise, and enable the access to automatic tallyingprocedures. Second, it multiplies collective decision-making-like situations: so-cial networks, recommender systems, and so on. Finally, computer networkshave considerably changed the way knowledge disseminates. This also applies tovoting: new citizens’ initiatives to promote alternative voting procedures appearevery day. They spread much faster and further than they used to, and as aconsequence they disseminate in some way ideas from social choice theory.

The second major breakthrough is the advent of mobile devices. Ubiquity hasa major impact on social choice. It now becomes possible to use computer-aidedcollective decision making in virtually every situation. More than that, it hasbrought connectivity and technology to a substantial part of the population thatwas before excluded from the technological sphere.

Social choice is facing a unique situation in its history. It is no longer lim-ited to the political and scientific communities, but it is now ready to be widelyapplied. The goal of this chapter is to analyse the major difficulties that socialchoice theorists will probably have to face if they want to apply social choice inpractice. The considerations proposed in this chapter are inspired by the activedevelopment of a web application dedicated to social choice. It is important tonote that this is by no means intended to be the description of scientific resultsobtained by a rigorous experimental approach. It can be seen as nothing morethan a compilation of lessons learned from the user experience with this plat-form. Moreover, this chapter voluntarily excludes the security issues posed byelectronic voting. We make the assumption that the collective decision makingsituations analysed here are uncritical situations (further referred to as low- andmiddle-stake situations) where the voters can tolerate some lack of guarantees interms of security, authentication and certification.

This chapter is organised as follows. First, we will quickly explain the main

3Except for the few countries — like Ireland — where people vote by ranking (or approving /disapproving) candidates for political elections.

Social Choice on the Web 3

features and objectives of the Whale web application in Section 20.2. Then, inSection 20.3, we will describe a particular use case and analyse the users’ be-haviour and impressions. Finally, in Section 20.4, we try to give more generallessons learned from the everyday use of the platform.

20.2 Whale: A Web Application for Social Choice

20.2.1 Goals and History

The genesis of Whale dates back to 2010 when we started to develop a small andlightweight web application dedicated to voting. At the beginning, this applica-tion was just intended to be an alternative to well-known and widely used pollapplications like Doodle® or Framadate. The initial observation was that suchapplications were very efficient tools to help solving collective decision makingproblems that happen in our everyday life (like choosing a restaurant, a can-didate for a position, a date for a meeting,...), but were sometimes too simple toadequately represent the complexity of the individuals’ preferences. For instance,most of these tools are limited to the expression of binary (Yes / No) preferences,but for some applications this is not enough. As an example, in situations wherea group of people has to hire someone for a job, it is reasonable to assume thatthe participants have more in mind than a simple dichotomy between approvedand disapproved candidates. When collectively choosing a date for a meeting,we are often faced with the situation where some date is not completely unavail-able to us, but is not ideal either, whatever it may mean. In such situations, itcould be useful to provide the participants with a way of expressing more subtlepreferences than just approving / disapproving alternatives.4

Even if there is a clear method to choose the winning option when we dealwith approval (Yes / No) ballots, namely the one which consists in choosing theoption that is approved by a maximal number of participants, this is not so clearanymore once the participants can choose among strictly more than two possibleevaluations for each option.5 In this case, there exist several collective decisionmaking procedures. All of them satisfy their own properties, but none is alwaysbetter than the others, so the procedure has to be carefully chosen, dependingon the context of the decision making problem.

Hence, as we can see, a lot of people use poll applications to make collectivedecisions nowadays, but these applications are sometimes too simple to ade-quately represent preferences and reflect the complexity of collective decisionmaking. On the other hand, social choice theory provides all the mathematicalbackground and models to make well-informed collective decisions. The mainrationale of Whale is to try to bridge the gap between these two worlds.

As of January 2017, four versions of Whale have been released. The mainreasons for this evolution are mainly technical: technologies tend to evolve morequickly in the context of web applications than in any other context. Developing

4The best evidence of this affirmation is that popular poll applications now propose to the partici-pants an intermediate level “(Yes)” between Yes and No.

5Or as soon as we are in the multiwinner case, which we exclude from the scope of this chapter.

4 S. Bouveret

an upgradable, maintainable, reusable and durable code is a real challenge andit is often easier to restart everything from scratch than to make the code evolve.

The first version of Whale was issued in 2010. At this time, it was just avery simple web application, developed in PHP with a Postgres backend for thepersistent data storage. The second version was developed by a team of studentsin 2012, using Java and the web framework Play (https://playframework.com/).Both versions had the same functionalities: registered users could create polls(date or classical) with several possible ballot types (approval, scores, rankingswith or without ties), users could vote, and the results given by several votingrules where displayed using a simple table (as in Figure 20.1).

Figure 20.1: Results of different voting procedures in the first version of Whale.

The third version of Whale was released in 2013, with the clear idea to focuson a better presentation of the results given by different voting rules. For that,we used data visualisation techniques, as we will see later. This version wasdeveloped in Scala / Java Servlet Pages and Postgres / JDBC for the backend,and the data visualisation was based on the D3.js library (https://d3js.org/).The last version of Whale (Whale 4) is in testing phase since September 2016.For this new version, the main focus was made on the simplicity of use and theresponsiveness of the application (adaptation to mobile devices). It was developedin Python with the Django framework (https://www.djangoproject.com/), is basedon the Bootstrap framework (http://getbootstrap.com/) for the responsiveness,and still on D3.js for data visualisation.

Social Choice on the Web 5

20.2.2 Features

Since the first version, the objectives of Whale have slightly evolved and thisapplication now has three main goals:

• to be used by non-specialists as a poll and voting platform;

• to be used by researchers and teachers as a pedagogical resource to illus-trate the concepts of voting;

• to be used by researchers as a source of real voting data for experiments.

Whale currently supports two kinds of decision problems, corresponding totwo different situations:

• open-ballot polls, where anyone can participate, and everyone can see thepreferences of the participants;

• sealed-ballot elections, which are accessible only at the invitation of thepoll’s creator. Each participant receives an individual 16-characters codethat must be used to connect to the poll. A voter can modify her vote anynumber of times until the poll closes. When the poll closes, the ballotsare publicly displayed, just hiding the names of the corresponding voters toguarantee anonymity.

Poll creators can currently choose between five ballot types: approval (Yes /No), scores (from 0 to 10), qualitative scale (--, -, 0, +, ++), rankings with ties,rankings without ties. Depending on the ballot type chosen, several methods areproposed to compute the collective preference and the election winner. Contraryto other voting platforms, the approach chosen was not to force the choice of aparticular voting rule, but rather to provide decision makers with all the toolsto understand the voters’ preferences, discuss them, and make enlightened de-cisions. Hence, the users can see at any time the results suggested by severalrelevant voting procedures. In the first two versions of Whale, these results weregiven as tables of scores (see Figure 20.1). However, we observed that these tableswere difficult to interpret, especially when the winners differed from one votingrule to another. In the subsequent versions of Whale, we have put a lot of effortinto developing data visualisation modules to present these results in an easilyunderstandable way. Four kinds of modules (illustrated in Figures 20.2, 20.3,20.4 and 20.5 on the best poster election use case presented in Section 20.3)dedicated to four kinds of voting rules have been developed so far.

• Histogram-based visualisations for scoring rules (see Figure 20.2, left), anda specific view dedicated to approval voting (Figure 20.2, right).

• Representations of the majority graph for Condorcet-consistent methods.Here, we use two different representations of the majority graph: a node-linkrepresentation where the thickness of a link encodes the margin information(see Figure 20.3 left), and an incidence matrix representation, with lines andcolumns ordered according to a particular score chosen among Copeland 0,Copeland 1 and maximin (see Figure 20.3 right). We also use colour to

6 S. Bouveret

encode the information about the margin (for a case of the matrix) or thescore of a candidate (for a node of the graph). This work on the majoritygraph has been extended to graph-compressed representations that seemto be quite efficient visualisation ways (Karanikolas et al., 2016), but thiswork has not been implemented in Whale yet.

• Run-off methods like plurality with run-off or Single Transferable Vote,where we simply display the list of candidates present in each round withthe score they obtain in this round (see Figure 20.4).

• Randomized cups, displayed exactly like the board of a sport cup havingcandidates as participants (see Figure 20.5).

Figure 20.2: Graphical view for scoring functions in Whale 4 (for the best posterelection). The left picture shows the Borda scores of the candidates. In the rightpicture, each curve corresponds to a candidate c and is a graphical representationof the function mapping a rank k to the k-approval score of c.

Beyond data visualisation techniques, the raw data of each poll can be ac-cessed through an HTTP API, and the raw (and anonymous) data of all the pollsof the database can be downloaded on the website. Currently three formats aresupported: CSV, JSON and Preflib (see Mattei and Walsh, 2013, and Chapter 15of this book). This data is provided for two main reasons. First, we hope thatresearchers will find valuable real-world data for their experiments. Second,providing an access to the data independently of any visualisation or aggrega-tion technique gives the opportunity to the developers to extend the visualisationmodules of Whale by developing their own representations of the voting profiles.

Other Existing Voting Platforms. Whale is certainly not the only online vot-ing platform project. If we exclude popular aforementioned poll applications, wecan cite two interesting and successful projects from academia: Pnyx (https://pnyx.dss.in.tum.de/) and Robovote (http://robovote.org/). Both also aim atbringing social choice theory to ordinary people by providing user-friendly col-lective decision making interfaces. Pnyx (Brandt et al., 2015) proposes severalways of expressing preferences (first-past-the-post, approval, ranking with ties,rankings without ties, pairwise comparisons), and several outputs are possible:

Social Choice on the Web 7

Figure 20.3: The majority graph and its adjacency matrix for the best posterelection. In the matrix, the rows and the columns are ordered by decreasingCopeland score.

Figure 20.4: Graphical view for run-off methods in Whale 4 (for the best posterelection). The picture shows the candidate ordering in the different rounds of theSingle Transferable Vote method.

8 S. Bouveret

Figure 20.5: Graphical view of randomized cups in Whale 4 (for the best posterelection).

unique winner, lottery, ranking without ties. Depending on the input and outputtypes, a different aggregation rule is used. Robovote (Caragiannis et al., 2017)adopts a different point of view. Preferences are always given as rankings, buta different aggregation rule is used according to whether these preferences aresupposed to be estimators of a ground truth (“objective” preferences) or totallysubjective. These two platforms have clearly a different objective from Whale, asthey impose a voting rule a priori, that the decision maker has to trust. In Whale,voting rules are more seen as voting profile exploration tools that give severalpoints of view on the voters’ preferences.

20.3 The Best Poster Award Use Case

After having introduced the main goals and features of the Whale web application,we will present in this section a situation where it has been used as a voting plat-form in a real context. This will be an occasion of discussing the main strengthsand weaknesses of the application by analysing the results and feedback receivedfrom the users after the election.

20.3.1 Organisation

The situation we will speak about in this section is the election of the bestposter in the main French-speaking conference in Geomatics, SAGEO’14 (https://sageo2014.sciencesconf.org/). The conference involved about 120 partici-pants, and 12 posters — referred to as Poster A to Poster L — were competingfor the best poster award. It is important to note that most of the participants(i.e., the pool of voters) were from the field of geography, mathematics or com-

Social Choice on the Web 9

puter science, but to the best of our knowledge none of them had a backgroundin collective decision making.

From a practical point of view, each participant was given — together with theconference kit distributed at the registration desk — a voting sheet containingthe following information:

• the direct URL to the voting page;

• a personal 16-characters voting code (randomly generated by Whale);

• the list of posters with labels (A...L), authors, titles.

The vote was opened during the first two days of the conference and was closedjust a few minutes before the session dedicated to the best poster award (whichshows the advantages of automatic tallying). People were free to vote using theirown laptops or smartphones, but could also use a computer which was providedby the organisers.

The voting process was presented during the opening session of the confer-ence. During this session, we presented the technical details concerning theconnection to the voting page and the voting procedure that would be used toelect the winner. The choice we made was to ask participants to give completerankings of the posters and to give the best poster award to the Condorcet win-ner if there is one, and to the Borda winner otherwise. This choice was madeboth for pedagogical and for simplicity purposes: the goal was mainly to promotealternative voting procedures which are easy to explain, to implement and to un-derstand, and have well-known and good properties. As we will see later, thischoice is debatable.

20.3.2 Feedback and Results

In the end, 61 persons took part to the vote, in other words, slightly more thanhalf of the attendees, which is a reasonable score. We collected some feedback onthe voting process by directly and informally discussing with the voters. Threekinds of remarks were made by several participants.

• Most people had never voted electronically before for this kind of elections,and had never used rankings to vote for an election. Most of them had notheard at all of the voting procedures we used before the presentation duringthe opening session. They seemed to be very happy to discover these kindsof methods, and had a real interest in alternative voting procedures. Fromthis point of view, the data visualisation, may they be simple and imperfect,were helpful tools to better understand how these procedures work.

• On the negative side, rather surprisingly, a common remark is that hav-ing to type a 16-characters code to access the personal voting page wasquite burdensome. This remark speaks for the use of light authenticationprocesses for non-critical settings, at the price of lower security guarantees(see Section 20.4.1).

10 S. Bouveret

• Concerning the voting procedure itself, a very important remark is that ask-ing for complete rankings was completely inappropriate in an election set-ting with 12 candidates where people usually have clear preferences overnot more than 4 or 5 candidates.

When the poll closed, we used Whale to tally the votes and determine thewinner of the best poster award. The majority graph of the election is shown inFigure 20.3.

It turns out that there was a Condorcet winner — Poster B — that should havebeen elected according to the voting rule chosen. However, if we have a closerlook at the majority margin matrix shown in Figure 20.3, we notice that PosterG is very close to being a Condorcet winner but is not because it loses againstPoster B with a margin of 3 votes (32 voters in favour of Poster B, 29 in favour ofPoster G).

Let us have a look at the winner given by the most classical scoring votingrules, which are not Condorcet-consistent. Poster B is the (co-)winner only for{5, 9, 10, 11}-approval (hence Veto), whereas Poster G wins for {2, 3, 4, 5, 6, 7, 8, 9}-approval and for Borda, and defeats Poster B for the plurality rule.

The settings where different voting rules yield different winners are not un-common, and are simply due to the fact that they are based on different nor-mative definitions of a democratic consensus and on different interpretations ofthe voters’ preferences. However, a basic analysis of the profile reveals that themajority in favour of Poster B might not be only due to a natural interpretationof the voters’ rankings.

Namely, if we look at the voting profile, we observe a clear bias in favour ofcandidates appearing earlier in alphabetical order. This bias seems to increase ascandidates approach from the end of the rankings. This impression is confirmedby a numerical analysis of the profile.6 Table 20.1 shows the normalized Kendall-Tau distance (that is, the number of pairwise disagreements divided by the totalnumber of pairs) between alphabetical orders and voters’ rankings for the k worstand the k best candidates in the rankings.

k 2 3 4 5 6δ (%) for the k worst 34.4 42.6 38.3 40.2 44.3δ (%) for the k best 57.4 51.4 49.7 49.8 49.5

Table 20.1: Normalized Kendall-Tau distance between alphabetical orders andvoters’ rankings for the k worst candidates and the k best candidates in therankings.

This table clearly shows that near the bottom of the ranking, the voters tendto agree more with alphabetical order than near the top. How does it apply toPoster B and Poster G? Table 20.2 shows the number of voters for which B and Gboth appear amongst the last k candidates of the ranking and preferring PosterB (resp. G) to Poster G (resp. B).

6This analysis just shows some informal trends, as a rigorous analysis would have required tostart from a hypothesis on the probability distribution of the votes and evaluate it as regards to thedata obtained during the election.

Social Choice on the Web 11

k 2 3 4 5 6 7 8 9 10 11 12#(B � G) 1 1 2 6 8 8 11 12 20 25 32#(G � B) 0 0 0 0 1 3 6 10 14 21 29Ratio (%) 0 0 0 0 11.1 27.3 35.3 45.5 41.2 45.7 47.5

Table 20.2: Number of voters preferring Poster B (resp. G) to Poster G (resp. B)when B and G both appear amongst the last k candidates in the rankings.

Since about 47.5% of the voters prefer Poster G to B in general, there is nospecial reason that this ratio changes a lot when the preferences are limited tothe bottom k ranks. Table 20.2 shows exactly the contrary. When B and G areboth at the bottom of the ranking, there is a much higher probability that G isranked after B (hence respecting alphabetical order) than the other way around.

The reasons of this obvious bias are pretty clear and are to be related tothe aforementioned feedback on the voting procedure: most people have clearpreferences on the first 4 or 5 candidates at most, but tend to order the rest ofthe candidates randomly.7 In Whale, the interaction effort required by addingcandidates at the end of the ranking is significantly lower if they are added in thesame order as they appear on the voting page (in alphabetical order in the case ofthe best poster election). That clearly explains the bias we observe at the bottomof the rankings. This phenomenon could have been anticipated, since this biashas already been clearly observed and measured by groups of researchers in thecontext of real political elections (see e.g. Krosnick, 2006; Ho and Imai, 2008).

We can learn at least two lessons from this analysis. First, where the contextpermits it, the voting interface should be designed so as to break any a prioriorder in the presentation of the candidates. A solution can be for instance toshuffle the candidate list every time a new voter accesses to the voting page.Secondly, when the number of candidates exceeds 4 or 5, one should absolutelyavoid asking for complete rankings, except in very specific contexts where eachvoter absolutely knows every candidate and needs to give a clear opinion on eachof them.

Epilogue. In the end, it was quite clear from the analysis of the voting profilethat even if Poster B was the Condorcet winner, it probably was for exogenousreasons that had nothing to do with the quality of the work perceived by the vot-ers, whereas there seemed to be strong arguments justifying the fact that PosterG is actually a better candidate than Poster B. We hence decided to manipu-late the election by changing the rules and giving the best poster award to bothPoster B and G. We carefully explained the reasons of this change to the voters.This change was possible because we were in a rather small community of well-intentioned candidates and voters. In a higher-stake context, things would havebeen very different and it would have been all the more important to carefullychoose the procedure and test the interface before proceeding to the election.

7This is not always the case. For political elections, for instance, we can reasonably assume thatpeople might have a clear idea about both the top and the bottom of the ranking, with a pool ofcandidates with unclear order in-between.

12 S. Bouveret

20.4 Lessons Learned from Ordinary Users

We have described in the previous section the feedback received from the partic-ipants to a real election having used Whale as a voting platform. In this sectionwe will now give some informal feedback received from people using Whale ona regular basis for everyday collective decision making. This feedback providesvery useful insights on how to improve the user experience on a poll / votingplatform. More importantly, it also gives directions and priorities for future re-search on social choice, by pointing to some practical problems that still needappropriate theoretical models and solutions.

20.4.1 Lighter is (Often) Better

The first important lesson learned from personal experience is that the successof a collective decision making platform like Whale largely depends on a carefulanalysis of its use cases and of its intended users. At the heart of this analysis, itis especially important to understand the stakes of the typical voting situationsthat the application will have to process. In the context of voting, we can typicallydistinguish three kinds of situations:

• high-stake elections like political elections, for which the requirements interms of confidentiality, verifiability, transparency and availability are sohigh (and attackers potentially so equipped) that they require specific toolswith mathematically provable properties8 — like Belenios for instance (seeCortier et al., 2013);

• middle-stake elections like local political elections or recruitment for a job,where some mild guarantees must be given on the transparency of the pro-cess and the anonymity of the ballots, but where it is acceptable for theusers to trust a third-party application and nobody has a high incentive toattack the voting server to manipulate the election;

• low-stake elections where the participants trust each other and the votingplatform is just there to help people express their preferences about a set ofoptions and discuss about them.

We can observe that most collective decision making we are faced with inour everyday life falls in the third category. In this case, it is crucial for theprocess to be as easy, light, and permissive as possible. These features are oftencontradictory with the standard point of view in voting that usually assumes that:(i) the set of candidates is fixed beforehand and does not evolve in time, (ii) votesare personal, and should not be altered or removed by anyone other than thevoter herself, and (iii) it is forbidden for the same person to vote several times.In traditional voting theory, any action contradicting these three assumptions isseen as a (malicious) manipulation, either by the voters or by the chair herself.Experience shows that in many situations, it is perfectly fine for the system toallow these actions, mostly because we are the context of small communities

8It is even doubtful that any form of electronic or online voting can guarantee these properties.

Social Choice on the Web 13

where participants all know and trust each other and have a strong incentive tofind a consensus that is beneficial to everyone. If the system is too restrictive,it can be observed that participants will try to tweak it to do what they want todo (for instance vote several times if they cannot modify votes, create new pollsif they cannot add / remove candidates...), or simply not use it and switch to amore permissive system.

These observations thus argue for of having two different sets of parameters:one dedicated to middle-stake elections that gives some mild guarantees on theverification of the usual assumptions (fixed set of candidates and voters, onevote for each voter, a vote cannot be altered...), and one dedicated to everydaysituations and permitting every action. This last situation has not been studiedso much by social choice theorists to the best of our knowledge, which shows afirst divergence between theory and what people really do in practice.

20.4.2 Preference Representation

The second divergence concerns preference representation. In practice, votingprocedures usually ask for one name on a ballot. There are many reasons forthat: it is cognitively simple, easily understandable, and simplifies tallying.

At the other end of the scale, social choice theory is usually based on theassumption that a voting procedure takes a profile of linear orders (completerankings) as input, and outputs either a collective ranking, a winner, or a setof co-winners. Hence, the traditional assumption is that each voter is able torank all the candidates. As we have seen in Section 20.3, this assumption is justcompletely unrealistic as soon as there are more than four or five candidates torank. As a result, the order given for the candidates at the bottom of the scale(or around the middle, depending on the context) is meaningless, or, worse, canbe strongly biased.

There are many intermediate ways of expressing preferences on a ballot. Theright level for most applications is probably to ask people to give scores to can-didates, scores being taken from a small qualitative or numerical scale (e.g.,{Yes,No}, {−1, 0, 1}, {0, . . . , 5}, {−−,−, 0,+,++},...). Even if a setting like approvalvoting is well understood and has been widely studied, this does not seem to bethe case for other score-based ballot settings,9 in spite of the obvious cognitiveinterest in this kind of ballots.

20.4.3 Incomplete Preferences

The third divergence that we can observe between theory and practice concernspotentially incomplete preferences. Whereas most work in voting theory assumesthat the voters express complete preferences, in the sense that they express aclear opinion (rank, score,...) on each candidate, we cannot rule out the possi-bility that a voter is unable to do so in practice, particularly when the number ofcandidates is high as in the example we have discussed in Section 20.3. More-over, as soon as we allow the set of candidates to be dynamic and candidates to

9Apart from work in experimental voting that analyses the behavioural differences induced bydifferent scoring scales.

14 S. Bouveret

be added during the election, as we have seen in Section 20.4.1, we need to beable to deal with incomplete ballots, since at the time we add new candidates, wedo not know yet the opinion on these candidates of the voters that have alreadyvoted.

There are two usual ways of dealing with incomplete preferences. The firstpossibility is to ignore them and force people to give an opinion for every singlecandidate (e.g., a complete ranking).

The second way of dealing with incomplete preferences is to use a kind ofballot with possible indifferences between candidates (e.g., rankings with ties,truncated rankings, approval ballots, ...) and use a default value for each can-didate not evaluated by the voter (for instance “No” for approval, not-ranked fortruncated rankings, 0 for the numerical scale {0, . . . , 5}).

None of these methods is satisfactory. We have seen earlier how bad the firstmethod can be. The second method can be seen as a workaround, but turns outto be really unsatisfactory in some contexts, because giving a default value to agiven candidate is different from the absence of information concerning this can-didate. Let us take for instance the voting situation considered by Karanikolaset al. (2016) about the election of the best movie among the ten most popularmovies of the last decades according to IMDB registered users. Most voters onlyhad a partial opinion about the movies, just because they had not watched themall. If we ask their opinion on a {0, . . . , 5} numerical scale, assuming that thedefault value is 0, we will simply not be able to distinguish between a movie theyhave not watched and a movie they have not liked at all.

These examples show how crucial it is to provide voters with a way not togive any opinion on some candidates. Otherwise, the process of voting will bepainful to them, and the result given by the voting procedure might be biasedor irrelevant. However, the ways to deal with incomplete ballots is still not clear.Collective decision making in this context probably comes down to the followingquestion: between a well-known candidate which is just moderately appreciatedby everyone and a candidate that just a few voters know but about which all ofthem are very enthusiastic, which one should be elected? The answer to thisquestion probably depends on the context, but incomplete preferences wouldprobably deserve a careful attention from the scientific community.

20.4.4 Multiple Voting Procedures

The final issue we will quickly discuss is a well-known difficulty related to the na-ture of social choice itself, and that is as old as the scientific discussion betweenJean-Charles de Borda and Nicolas de Condorcet about the respective meritsof different voting systems. As soon as there are more than two candidates tochoose from and we ask the voters for more than approval ballots, there are sev-eral ways of electing the winner of the election, all corresponding to a differentnotion of consensus. Worse, there are extreme cases where each candidate is de-clared winner of the election by a different voting rule. The problem of choosingthe right voting rule, whatever it means, is not simple in this case.

There are several possible approaches to this problem. First, we can imposea voting rule a priori and choose the winner according to this voting rule. This is

Social Choice on the Web 15

the approach used by several voting platforms.10 We could also think of lettingelection chairs (or even the voters themselves, see Laslier, 2012) choose amongstseveral proposed voting rules before the election begins. The real question hereis to provide enough explanation to ensure that stakeholders make an informeddecision about which voting rule they choose to use. Several recent works insocial choice have started to study this question of making people choose thevoting rule that most fits their needs (see, e.g., Cailloux and Endriss, 2016).

In the aforementioned context of low-stake elections, another point of viewis possible. Since in this context the voting platform is mainly used to elicitpreferences and discuss about them, voting rules can be seen as many ways ofaggregating the preference information contained in the profile and as many waysof exploring this information. This is the approach we have followed in the firsttwo versions of Whale, where the raw results given by several voting rules weredisplayed to the user, as shown in Figure 20.1. However, as we might expect,simply displaying the potentially contradictory information given by the differentvoting rules does not help much making a decision, and people were often lostespecially when different voting rules yield different winners.

For the last two versions of Whale, we have tried a different approach basedon a literal interpretation of “exploring the voting profile”. Namely, we have triedto use information visualisation techniques to provide interactive visualisation ofthe results given by voting procedures. This has lead to the set of basic graphicalviews that are proposed in Whale 3 and 4 (see Figures 20.2, 20.3, 20.4 and 20.5).This work is just at the beginning, but we believe that it is a promising approach,both to better understand and analyze the voting profiles, and as pedagogicaltools to explain to laypersons how the voting procedures work.

20.5 Conclusion

In this chapter, we have presented a web application dedicated to collective de-cision making. The aim was not exactly to advertise a particular platform, butrather to put into light some practical problems that have not received a lot ofattention by the scientific community yet, or, at least, would deserve more at-tention. Amongst these topics, the design and analysis of voting rules that areusable in practice for everyday collective decision making seems to be of the ut-most importance. That concerns the analysis of voting rules that work on smallscales of scores rather than on linear orders and also voting rules that are robustto the absence of information concerning some candidates. It would be also cru-cial to put some efforts into making voting rules be more than only black boxesor oracles that elect a winner for a given set of ballots. It could mean developingways of explaining these voting rules and providing people with concrete meansto argue to choose the right one, or use them as voting profile exploration tools,through graphical visualisation for instance. It is a safe bet that some of theproblems listed above will be amongst the future trends of the discipline.

10And also to some extent by poll platforms like Doodle® or Framadate, that recommend to chooseamongst the set of co-winners lexicographically maximising the number of “yes” and the number of“(yes)”.

16 S. Bouveret

Acknowledgments

Most work about information visualization presented in this chapter has beenmade in collaboration with Renaud Blanch. Several people have contributed toWhale, especially Marie-Jeanne Natete, which was the main developer of Whale 4.Whale is supported by ANR project CoCoRICo-CoDec.

Bibliography

F. Brandt, G. Chabin, and C. Geist. Pnyx: A powerful and user-friendly tool forpreference aggregation. In Proceedings of the 14th International Conference onAutonomous Agents and Multiagent Systems (AAMAS), pages 1915–1916, 2015.

O. Cailloux and U. Endriss. Arguing about voting rules. In Proceedings of the15th International Conference on Autonomous Agents and Multiagent Systems(AAMAS), pages 287–295, 2016.

I. Caragiannis, S. Nath, A. D. Procaccia, and N. Shah. Subset selection via im-plicit utilitarian voting. Journal of Artificial Intelligence Research, 58:123–152,2017.

V. Cortier, D. Galindo, S. Glondu, M. Izabachene, et al. A generic constructionfor voting correctness at minimum cost-application to helios. IACR CryptologyEPrint Archive, 2013:177, 2013.

D. E. Ho and K. Imai. Estimating causal effects of ballot order from a randomizednatural experiment the California alphabet lottery, 1978–2002. Public OpinionQuarterly, 72(2):216–240, 2008.

N. Karanikolas, R. Blanch, and S. Bouveret. Edge-compressed majority graph:Where social choice meets information visualization. In Proceedings of the SixthInternational Workshop on Computational Social Choice (COMSOC), Toulouse,France, June 2016.

J. A. Krosnick. In the voting booth, bias starts at the top. The New York Times,November 2006.

J.-F. Laslier. And the Loser Is. . . Plurality Voting, pages 327–351. Springer BerlinHeidelberg, Berlin, Heidelberg, 2012.

N. Mattei and T. Walsh. Preflib: A library of preference data HTTP://PREFLIB.ORG.In Proceedings of the 3rd International Conference on Algorithmic Decision The-ory (ADT 2013), Lecture Notes in Artificial Intelligence. Springer, 2013.