HOW SHOULD WE SCORE ATHLETES AND CANDIDATES: GEOMETRIC SCORING RULES

ALEKSEI Y. KONDRATEV, EGOR IANOVSKI, AND ALEXANDER S. NESTEROV

Abstract. We study how to rank candidates based on individual rankings via scoring rules. Each position in each individual ranking is worth a certain number of points; the total sum of points determines the aggregate ranking. Our selection principle is consistency: once one of the candidates is removed, we want the aggregate ranking to remain intact. This principle is crucial whenever the set of the candidates might change and the remaining ranking guides our actions: whom should we interview if our first choice got a better offer? Who gets the cup once the previous winner is convicted of doping? Which movie should a group watch if everyone already saw the recommender system’s first choice? Will adding a spoiler candidate rig the election? Unfortunately, no scoring rule is completely consistent, but there are weaker notions of consistency we can use. There are scoring rules which are consistent if we add or remove a unanimous winner – such as an athlete with suspiciously strong results. Likewise, consistent for removing or adding a unanimous loser – such as a spoiler candidate in an election. While extremely permissive individually, together these two criteria pin down a one-parameter family with the scores 0, 1, 1 + p, 1 + p + p2,.... These geometric scoring rules include Borda count, generalised plurality (medal count), and generalised antiplurality (threshold rule) as edge cases, and we provide elegant new axiomatisations of these rules. Finally, we demonstrate how the one-parameter formulation can simplify the selection of suitable scoring rules for particular scenarios.

Keywords: rank aggregation, scoring system, ranking system, rating system, points system, sports ranking, sports rating, medal count, lexicographic ranking, Borda count, indepen- dence of irrelevant alternatives, group recommendation, least misery arXiv:1907.05082v2 [cs.GT] 15 Dec 2019

Aleksei Y. Kondratev  corresponding author  National Research University Higher School of Economics, 16, Soyuza Pechatnikov st., St. Petersburg, 190121, Russia; Institute for Regional Economic Studies RAS, 38, Serpuhovskaya st., St. Petersburg, 190013, Russia  [email protected]  http://orcid.org/0000-0002- 8424-8198 Egor Ianovski  National Research University Higher School of Economics, 16, Soyuza Pechatnikov st., St. Petersburg, 190121, Russia  [email protected] Alexander S. Nesterov  National Research University Higher School of Economics, 16, Soyuza Pechatnikov st., St. Petersburg, 190121, Russia  [email protected]  http://orcid.org/0000-0002-9143-2938 We thank Herv´eMoulin, and other our colleagues from the HSE St. Petersburg Game theory lab for their suggestions and support. We also are grateful to Josep Freixas for providing us with the MotoGP 1999 example during the 15th European Meeting on Game Theory (SING15). 1 Introduction The 2014-2015 IBU Biathlon World Cup took place over a series of races in the winter of 2014/2015. At the end of each race the competitors were ranked in the order they finished, and assigned a number of points based on their position in that order. The scores for the first thirteen positions are 60, 54, 48, 43, 40, 38, 36, 34, 32, 31, 30, 29, 28. The scores the athletes won are summed across the races, and the one with the highest score is declared the winner – a procedure that is known as a scoring rule or a points system. The Women’s Pursuit category consists of seven races. Kaisa M¨ak¨ar¨ainencame first with two first place finishes, two second, a third, a fourth, and a twelfth, for a total score of 348 points. Second was Darya Domracheva, with four first place finishes, one fourth, a seventh, and a thirteenth, for a total score of 347. In tenth place was Ekaterina Glazyrina, well out of the running with 190 points. These results are given in Table 1.

Table 1. Women’s Pursuit results of the 2014-2015 Biathlon Athlete 1 2 3 4 5 6 7 Total M¨ak¨ar¨ainen 60 60 54 48 54 29 43 348 Domracheva 43 28 60 60 60 36 60 347 . . Glazyrina 32 54 10 26 38 20 10 190

Four years later, Glazyrina was disqualified for doping violations, and all her results from 2013 were annulled. This bumped Domracheva’s thirteenth place finish in race two into a twelfth, and her total score to 348. The number of first place finishes is used as a tie breaker, and in March 2019 it was announced that M¨ak¨ar¨ainenwill be stripped of the Small Crystal Globe trophy in favour of Domracheva. Because the tenth place competitor was disqualified for doping four years after the fact.1 Clearly there is something unsatisfying about this. We would hope that the relative ranking of M¨ak¨ar¨ainenand Domracheva depends solely on the relative performance of the two athletes, and not on whether or not a third party was convicted of doping, especially if said third party was not a serious contender for the title. In this paper we will discuss a few classical results from social choice theory which describe why this is impossible, and suggest the best possible course of action left to us given the mathematical impossibilities we face. This involves introducing two extremely weak inde- pendence axioms, which turn out to characterise a one-parameter family of scoring rules – the geometric scoring rules.

1Eventually the IBU decided to award both athletes the first place: https://biathlonresults.com 3

The science of impossibility

Suppose we have a set of m athletes and the results for n races, R1,...,Rn. Ri is the order in which the athletes finished race i. What we are after is a procedure which will transform the n race results into a single ranking for the entire competition, R: a function

(R1,...,Rn) 7→ R. We are not looking for an arbitrary function, but the end result R must reflect the results of the individual races in some way. A minimal condition is unanimity – if an athlete finishes first in every race, we should expect this athlete to rank first in R. Motivated by the scenario in the introduction, we also want the relative ranking of athletes a and b in the end result to depend only on the relative ranking of a and b in the individual races, a condition known as the independence of irrelevant alternatives (Arrow, 1950; Nash, 1950) and the consistency principle (Thomson, 2011). Here we hit the most famous result in social choice theory – Arrow’s result that the only function that meets our criteria is dictatorship (Campbell and Kelly, 2002). Now, social choice theory is prone to colourful nomenclature, so it is important to underline that the characteristic feature of dictatorship is not jackboots and moustaches, but that all decisions stem from a single individual. In the case of a sporting competition, this could be the case where the first n − 1 races are treated as warm-ups or friendly races, and only the

finals Rn contributes to the final ranking R. This is not necessarily an absurd system, but it will not do if we want to keep viewers interested over the course of the event, rather than just the finals. We need to relax independence. A weaker independence condition we may consider is independence of winners/losers.2 As the name suggests, this is the condition that if we disqualify the top or the bottom athlete in the final ranking R, the remainder of the ranking remains unchanged. This could be a pressing issue if the winner is accused of doping, and a rule that satisfies this condition will guarantee that the cup is given to the runner up without requiring a retallying of the scores. In the case of the loser, there is the additional concern that it is a lot easier to add a loser to a race than a winner, and if the authors were to take their skis off the shelf and lose ingloriously in the next Biathlon, one would hope that the standing of the real competitors would remain unaffected. It turns out that, given some standard assumptions, there is a unique rule that satisfies this – the Kemeny rule (Kemeny, 1959; Young, 1988). The procedure amounts to choosing a ranking R that minimises the sum of the Kendall tau distance from R to the individual races. This is one of its disadvantages – the definition is clear to a mathematician, but not to anyone else. Sporting fans will be asked to accept the results produced by what is essentially a black box. What’s worse, it is a difficult procedure computationally (Bartholdi et al.,

2Independence of winners/losers is also known as local independence of irrelevant alternatives. Independence of losers is also known as independence of bottom alternatives (Freeman et al., 2014). 4

1989), so even working out the winner may not be feasible. But perhaps most damning of all is that it violates a property known as electoral consistency. Suppose a biathlete comes first in every category of the championship (sprint, pursuit, and so on). It would be very strange if she failed to come first overall. Under the Kemeny rule, we would have to live with such strangeness. In fact, the only rules that guarantee electoral consistency are the scoring rules (Smith, 1973; Young, 1975). It seems there is no alternative to the rules actually used in Biathlon, Formula One, the Tour de France, and other events in this format – but that is not necessarily a bad thing. Scoring rules are easy to compute and understand, and every additional result contributes to the overall ranking in a predictable way, all of which is very desirable for a sporting event. We have seen that neither of the independence notions we have defined so far can apply here, but how bad can the situation get? The answer is, as bad as possible. A theorem of Fishburn (1981) shows that for any monotone decreasing series of scores whatsoever, it is possible to construct a sequence of race results such that if one athlete is removed the remaining order is not only changed, but inverted.3 So while M¨ak¨ar¨ainenmay not be pleased with the current turn of events, there is a possible biathlon where after the disqualification of Glazyrina M¨ak¨ar¨ainenfinished last, and Domracheva second to last. It is interesting to speculate whether the competition authorities would have had the resolve to carry through such a reordering if it had taken place.

Main result In order to motivate our final notion of independence, let us first consider why the re- sults of the biathlon may not be as paradoxical as they appear at first glance. Note that removing Glazyrina from the ranking in Table 1 changed the score of Domracheva but not of M¨ak¨ar¨ainen. This is because M¨ak¨ar¨ainenwas unambiguously better than Glazyrina, fin- ishing ahead of her in every race, while Domracheva was beaten by Glazyrina in race 2. As such, the athletes’ performance vis-`a-visGlazyrina served as a measuring stick, allowing us to conclude that M¨ak¨ar¨ainenwas just that little bit better. Once Glazyrina is removed, however, the edge M¨ak¨ar¨ainenhad is lost. So suppose then that the removed athlete is symmetric in her performance with respect to all the others. In other words she either came last in every race, and is thus a unanimous loser, or came first, and is a unanimous winner. Surely disqualifying such an athlete cannot change the final outcome? Why, yes it can. The results for the Women’s Individual category of the 2013-2014 IBU Biathlon World Cup are given in the left panel of Table 2. The event consists of two races, and Gabriela

3Saari (1989) showed that not only inversions but any permutations can happen. 5 Table 2. Women’s Individual results of the 2013-2014 Biathlon Athlete 1 2 Total Athlete 1 2 Total Soukalov´a 60 60 120 Soukalov´a 60 60 120 Domracheva 38 54 92 Domracheva 40 60 100 Kuzmina 54 30 84 Kuzmina 60 31 91 Skardino 36 36 72 Hildebrand 29 48 77 Hildebrand 28 43 71 Skardino 38 38 76

Notes: The left panel presents the official results; the right panel presents the results after a hypothetical disqualification of Soukalov´a.The scores given in the table are before the disqualification of another athlete, Iourieva, that occurred a few months after the race. With the most recent scores we would still observe Hildebrand overtaking Skardino, but we would have to resort to tie-breaking to do it.

Soukalov´acame first in both, and is thus a unanimous winner, followed by Darya Dom- racheva, Anastasiya Kuzmina, Nadezhda Skardino, and Franziska Hildebrand. However, in the hypothetical event of Soukalov´abeing disqualified the result is different: the recalcu- lated scores are in the right panel of Table 2. Domracheva takes gold and Kuzmina silver as expected, but Hildebrand passes Skardino to take the bronze. However, in contrast to the previous paradoxes, this is one we can do something about. By picking the right set of scores we can ensure that the unanimous loser will come last, the unanimous winner first, and dropping either will leave the remaining order unchanged.

Let us formalise our two axioms. An athlete is a unanimous loser just if the athlete is ranked last in every race. A scoring rule satisfies independence of unanimous losers if the unanimous loser is ranked last in the overall ranking, and removing the unanimous loser from every race leaves the overall ranking of the other athletes unchanged. Symmetrically, an athlete is a unanimous winner when ranked first in every race. In- dependence of unanimous winners is satisfied if the unanimous winner is ranked first in the overall ranking, and removing the unanimous winner from every race leaves the overall ranking of the other athletes unchanged. Next, note that hitherto we have not been precise about how we go about recalculating

the scores when an athlete has been dropped. The issue is that if we use the values s1, . . . , sm to score m athletes, if we remove an athlete we are left with m scores and m − 1 athletes.

In principle we need another set of t1, . . . , tm−1 scores to score m − 1 athletes, and indeed a separate vector of k scores to score any k.4 In practice most sporting events simply obtain the ti values by trimming the first sequence, i.e. t1, . . . , tm−1 = s1, . . . , sm−1, and sm is dropped. Observe that the order produced by a scoring rule is invariant under scaling and transla- tion, e.g. the scores 4, 3, 2, 1 produce the same order as 8, 6, 4, 2 or 5, 4, 3, 2. We will thus

4The reader may find the formal definition in the appendix. 6

say that scores s1, . . . , sm and t1, . . . , tm are linearly equivalent if there exists an α > 0 and

a β such that si = αti + β.

The intuition behind the following result is clear: t1, . . . , tk produces the same ranking of the first/last k athletes, if and only if it is linearly equivalent to the first/last k scores in the original system.

m m Theorem 1. Suppose a scoring rule uses scores s1 , . . . , sm to score m athletes. The scoring m m rule satisfies independence of unanimous losers if and only if s1 > . . . > sm and the scores for k k m m k athletes, s1, . . . , sk, are linearly equivalent to the first k scores for m athletes, s1 , . . . , sk , for all k ≤ m. m m The rule satisfies independence of unanimous winners if and only if s1 > . . . > sm and k k the scores for k athletes, s1, . . . , sk, are linearly equivalent to the last k scores for m athletes, m m sm−k+1, . . . , sm, for all k ≤ m.

Now we see why the biathlon scores are vulnerable to dropping unanimous winners but not unanimous losers – since the scores for a smaller number of athletes are obtained by trimming the full list, every subsequence of the list of scores is indeed equivalent to itself. However if we drop the winner, then the subsequence 60, 54, 48, ... , is certainly not linearly equivalent to 54, 48, 43, ... . What happens when we combine the two conditions? Given that we need not distinguish scores up to scaling and translation we can assume that the score for the last place, sm, is zero, and sm−1 = 1. The third-to-last athlete must then get a larger number of points than 1, say 1 + p. Now we have a sequence 0, 1, 1 + p, and we know it must be linearly equivalent to the sequence 1, 1 + p, sm−3. Since the only way to obtain the second sequence from the 2 first is to scale by p and add 1, it follows that sm−3 = 1 + p + p and so on. The formula for m−j the score of the jth position is sj = (p − 1)/(p − 1). This gives us the following family of scoring rules.

Theorem 2. A scoring rule satisfies independence of unanimous winners and losers if and only if it is a generalised geometric rule. That is, it is defined with respect to a parameter p, and the score of the jth position is linearly equivalent to:

 m−j p p > 1,  sj = m − j p = 1,  1 − pm−j p < 1.

m m Observe that the axioms we used are extremely weak individually. If s1 , . . . , sm is any m−1 m monotone decreasing sequence of scores whatsoever, and we obtain si by dropping sm, m−1 we will satisfy independence of unanimous losers. Likewise, if we obtain si by dropping 7 Table 3. Top 3 drivers’ results of the 1999 MotoGP 125cc

Athlete 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Total Alzamora 20 16 16 16 10 20 13 16 20 10 13 20 1 16 20 227 Melandri 10 20 16 8 11 25 25 25 25 16 20 25 226 Azuma 25 25 25 13 9 25 25 10 4 6 11 2 10 190

m s1 , we will satisfy independence of unanimous winners. In short our only restriction is that more points are awarded for the ith place than for the (i + 1)th place, which in the context of a sporting event is hardly a restriction at all. If we want to satisfy both axioms, however, we are suddenly restricted to a class with just one degree of freedom. This one-parameter class includes three famous rules as boundary cases. As p approaches infinity, the first position dominates all the others and we have the generalised plurality rule, also known as the lexicographic ranking or the Olympic medal count (Churilov and Flitman, 2006). With p = 1 we have the Borda rule, proposed by one of the founders of the mathematical study of elections (de Borda, 1781; McLean and Urken, 1995). As p approaches 0, we have the generalised antiplurality rule, also known as the threshold rule (Aleskerov et al., 2010). In the remainder of this paper we will explore this class. We shall see how our axioms allow new axiomatisations of well-known scoring rules, and how the one-parameter characterisation simplifies the search for an appropriate scoring rule for a situation.

p > 1: convex rules and winning in every race The FIM motorcycle Grand Prix is another event that uses a scoring system to select a winner. The scores are 25, 20, 16, 13, 11, 10, ..., 1, 0, ..., 0. The 125cc category of the 1999 season had a curious outcome: the winner was Emilio Alzamora, who accumulated the largest amount of points, yet did not win a single race (Table 3). This does not detract in any way from Alzamora’s achievement – he outperformed his competitors by virtue of his consistently high performance (compare with Melandri who performed well in the second half, and Azuma in the first), and if he did not take any unnecessary risks to clinch the first spot then he was justified in not doing so. However, racing is a spectator sport. If a fan attends a particular event then they want to see the athletes give their best performance on the day, rather than play it safe for the championship. Slow and steady may win the race, but blood and explosions wins viewer ratings. Bernie Ecclestone, the former chief executive of the Formula One Group, was outspoken about similar issues in Formula One – “It’s just not on that someone can win the world championship without winning a race.”5 Instead of the scores then used, Ecclestone proposed

5https://web.archive.org/web/20191116144110/https://www.rte.ie/sport/motorsport/2008/ 1126/241550-ecclestone/ 8 a medal system. The driver who finished first in a race would be given a gold medal, the runner-up the silver, the third the bronze. The winner of the championship would be the driver with the most gold medals; in case of a tie, silver medals would be added, then the bronze, then fourth-place finishes, and so on. In other words, he proposed the generalised plurality system with p = ∞. And indeed, for every other geometric scoring rule, it is possible to construct a profile where the overall winner did not win a single race.

Proposition 1. For any p < ∞, there exists a profile where the overall winner does not come first in any race.

There is a natural dual concept to Ecclestone’s criterion: rather than asking how many races an athlete must win to have a chance of winning the championship, we could ask after how many victories is the championship guaranteed. This leads us to the majority criterion, which requires that any athlete that won more than half the races should also win the championship. The majority criterion together with our independence axioms allows us to characterise generalised plurality.

Theorem 3. Generalised plurality is the only generalised scoring rule satisfying the majority criterion and independence of unanimous winners. Symmetrically, generalised antiplurality is the only generalised scoring rule that satisfies independence of unanimous losers and always ranks the majority loser last.

The presence of the majority criterion in the above theorem is not surprising. We could expect as much given the axiomatisation of plurality (Lepelley, 1992; Sanver, 2002). But the fact that adding independence of unanimous winners allows us to pin down generalised plurality is interesting because generalised scoring rules are known to be notoriously hard to characterise (Bossert and Suzumura, 2018), but in this case two intuitive axioms suffice. Here we run into a conundrum. Ecclestone’s criterion is desirable in the case of a sporting event for the reasons we have mentioned – the stakes are high in every race, and encourages the athletes to fight for the top spot rather than settling for second place. The majority criterion, on the other hand, tells a different story. If a driver were to win the first bn/2 + 1c races, the championship is over. He can lose his car keys, drive the track backwards, turn up drunk (no need to worry about setting a bad example for the fans, because the stadium will be empty), and still be guaranteed the first spot on the podium. One could argue that such an even is extremely unlikely, but the same argument applies if we enter the last race of the championship and one driver has a lead of two first-place finishes. What could have been the decisive moment of the championship becomes a write-off. 9

100 2010 – Present

2003 – 2009 80 1991 – 2002

60 1961 – 1990

1960 40 1950 – 1959

20 Geometric, p = 1.8

Geometric, p = 1.2 0

1st 2nd 3rd 4th 5th 6th 7th 8th 9th 10th

Figure 1. Scores used in FIA Formula One compared with geometric scores Notes: Scores for first place were normalised to 100. Blue dash p=1.2, brown dash p=1.8, orange round dots 1950–1959, black long dash 1960, green short dash 1961–1990, light blue long dash dot 1991–2002, red long dash two dots 2003–2009, purple solid 2010–present.

It seems there is a trade-off between keeping the tension high in an individual race and over the course of the entire tournament, which could explain why the organisers of Formula One went through so many scoring systems over the years (Figure 1).6

p = 1: a new characterisation of Borda Saari and Barney (2003) recount an amusing anecdote to motivate an axiom known as reversal symmetry. In a departmental election the voters were asked to rank three candidates. All voters ranked the candidates from best to worst, but the chair expected the votes to be ordered from worst to best. The “winner” was thus the candidate that ranked highest in terms of the voters’ assessment of unsuitability, rather than suitability for the role. After the ensuing confusion, the votes were retallied... and the winner was unchanged. The same candidate was judged to be at once the best and worst for the role. Presumably we ought to first give him a raise and then fire him. Saari’s story ended with the chair being promoted to a higher position, but of course in a sporting context the resolution would be a bit different: the fans would storm the judges’ stand and burn down the stadium. The relevant axiom here is reversal symmetry, which states that if an athlete a is the unique winner with race results R1,...,Rn, then if we invert the results of every race then a will no longer be the unique winner. Note that the axiom asks for less than one might expect

6https://web.archive.org/web/20100106134601/http://www.fia.com/en-GB/mediacentre/ pressreleases/f1releases/2009/Pages/f1_medals.aspx 10

– we are not asking that the athlete formerly judged the best is now judged the worst, but merely that the same athlete cannot be the best in both cases.

By itself, this axiom is quite weak. If we, without loss of generality, set s1 = 1 and sm = 0,

then the only restriction is that for 1 ≤ i ≤ m/2, sm−i+1 = 1 − si (Saari and Barney, 2003; m−2 Llamazares and Pe˜na,2015). In other words, we have b 2 c degrees of freedom. However, once we add either one of our axioms, we get Borda uniquely.

Theorem 4. Borda is the unique scoring rule satisfying reversal symmetry and one of independence of unanimous winners or independence of unanimous losers.

Within the context of geometric scoring rules, there is an easier way to see that Borda is the unique rule satisfying reversal symmetry – it is easy to see that the results produced by a rule with parameter p would be the inverse of the results produced by using a rule with parameter 1/p on the reversed profile. Since 1 = 1/1, the effect of running Borda on the reversed profile would be to reverse the resulting order.

p < 1: concave rules and minimising misery The astute reader may complain that we had promised to demonstrate how the one- parameter formulation of geometric scoring rules simplifies the task of choosing a scoring rule for a problem, yet so far our results have only concerned the edge cases of p = ∞, p = 1 or p = 0. To an extent this is unavoidable with a theoretical approach – if we set out to maximise something in the abstract, without focusing on the specifics of a particular application, we would expect to get precisely the edge cases.7 However, such a rationalisation cannot help but be unsatisfactory. To address this we will present an example of choosing a p in the context of group recommender systems. Many group recommender systems are based on the principle of least misery: the idea that a group is as happy as its unhappiest member (O’Connor et al., 2001; Masthoff, 2004, 2015; McCarthy et al., 2006; Jameson and Smyth, 2007; Amer-Yahia et al., 2009). The logic here is that while members of a group understand that they cannot all watch their favourite film, if any member ends up watching a film they consider absolutely terrible they will leave the group and do something else. If we want to keep the group together we want to target the unhappiest member, and clearly we can achieve this aim by setting p = 0. Generalised antiplurality will select the film with the least number of last positions, hence minimise the number of people who are maximally unhappy. In the case of a tie, the number of second-to-last positions are used as a tie-breaker, and so on.

7The cases where the optimal scoring rule was found to be different from Borda, plurality, and antiplurality are rare indeed. Some examples include Cervone et al. (2005), Lepelley et al. (2018), Diss et al. (2019) and Kamwa (2019). 11 Table 4. Maximum level of misery of the unhappiest agent

n = 2 n = 3 n = 4 n = 5 m 10 25 50 100 10 25 50 100 10 25 50 100 10 25 50 100 p ≤0.2 6 13 26 51 7 17 34 67 8 19 38 76 9 21 41 81 0.3 6 13 26 51 7 17 34 67 8 20 38 76 9 21 41 81 0.4 6 13 26 51 8 18 34 68 9 20 39 76 9 21 41 81 0.5 6 13 26 51 8 18 35 68 9 20 39 76 9 21 41 81 0.6 6 14 26 51 8 18 35 68 9 21 39 77 10 22 42 82 0.7 7 14 27 52 9 19 36 69 10 21 40 78 10 23 43 83 0.8 7 15 28 53 10 21 37 71 10 23 42 79 10 24 44 84 0.9 8 18 31 57 10 24 42 76 10 25 47 84 10 25 49 89 1.0 10 25 50 100 10 25 50 100 10 25 50 100 10 25 50 100

Notes: The table contains the bounds on the level of misery experienced by a group with n agents, choosing from m films, using a scoring rule with parameter p.

On the other hand, experiments have suggested that a least misery approach has poor utilitarian properties, while Borda performs quite well (Masthoff, 2015). So we seem to have a trade-off between misery and efficiency, which in the context of a geometric scoring rules amount to choosing a concave rule with 0 ≤ p ≤ 1. Which p to choose? In Table 4 we present the bounds on the level of misery (how the chosen alternative is ranked by the unhappiest agent) experienced in a group with varying levels of p. Based on these numbers we can see that choices of p between 0.5 and 0.8 offer misery guarantees close to the optimum, without straying as far from the utilitarian properties of p = 1.

Conclusion Scoring rules are omnipresent. They are used for hiring, in group recommender systems (Masthoff, 2015), web applications, such as meta-search engines, multi-criteria selection, word association queries (Dwork et al., 2001), in sports competitions, such as auto racing, cycling, biathlon, and even for aggregating results from gene expression microarray studies (Lin, 2010). Many countries use scoring rules in political elections: most of them use plurality, while Slovenia, Nauru and Kiribati use non-plurality scores (Fraenkel and Grofman, 2014; Reilly, 2002). It is likely that scoring rules are popular because of their simplicity. Yet the task of choosing a specific scoring rule for a given problem is a daunting one, as one must choose m numbers more or less arbitrarily. At the same time, for each of these applications consistency – independence of irrelevant alternatives – is a desirable property: we want the aggregate ranking to remain intact if the set of the alternatives changes. Unfortunately, no scoring rule 12

(or, indeed, any non-trivial rule) is completely consistent. To get a positive result, we intro- duce two relaxations: independence from removing the unanimous winner and independence from removing the unanimous loser.8 The practical relevance of our paper is that if the reader accepts that our two axioms are desirable – and they are very natural axioms – then the problem is reduced to choosing a single parameter p (Theorem 2). To our knowledge, in social choice literature this is the first characterisation not of a specific scoring rule, but of a non-trivial family.9 This family is sufficiently broad: not only does it include a continuum of convex and concave scores, but also three of the most popular scoring rules: the Borda count, generalised plurality (medal count) and generalised antiplurality (threshold rule). We demonstrate how the choice of the parameter p is constrained by the presence of other desirable axioms. The majority winner and majority loser criteria pin down the generalised plurality and antiplurality rules (Theorem 3). Reversal symmetry pins down the Borda rule (Theorem 4). We also discuss how to choose p for particular applications. Sports competitions call for moderately convex scores to balance incentives to win in each particular race and to perform well in all races (Proposition 1, Theorem 3). Recommendations for a group call for moderately concave scores in order to maximize overall welfare while limiting the misery of the unhappiest member of the group (Table 4). The optimal choice of p, of course, is going to depend on the particular demands of a given scenario. Future research can be devoted to finding desirable properties or specific methods that pin down a particular parameter p. Such methods could include data envelopment analysis (Green et al., 1996), statistical (Cervone et al., 2005), and worst-case analysis of a desirable property (Kondratev and Nesterov, 2019). Relaxing independence can be useful in other settings using scoring rules such as prefer- ential multiwinner voting (Skowron et al., 2019). In the weighted version of approval-based multiwinner voting (Thiele, 1895; Janson, 2018; Brill et al., 2018; Lackner and Skowron, 2019), if we will apply independence of unanimously approved candidates (analogous to our independence of unanimous winners), we will obtain geometric sequences of scores which include the top-k rule and a refinement of the Chamberlin–Courant rule as particular cases. Nor are these questions relevant for scoring rules only – one might think that no reasonable rule would violate something as weak as independence of unanimous losers, but Nanson’s rule (Nanson, 1882; McLean and Urken, 1995; Felsenthal and Nurmi, 2018) does exactly that.

8While there have been many relaxations of independence in the literature (independence of Pareto- dominated alternatives (Fishburn, 1973), independence of Smith-dominated alternatives, and independence of clones (Tideman, 1987)), for scoring rules they led to positive results only for plurality: it is the only scoring rule that satisfies independence of Pareto-dominated alternatives (Richelson, 1978; Ching, 1996) and weaker versions of independence of clones (Ozt¨urk,2018). 9See Chebotarev and Shamis (1998) for an extensive overview of previous axiomatisations of scoring rules. 13

A dual to this question is not what rules independence axioms lead to, but which axioms characterise geometric scores in different settings. The work of Aggarwal (2015) considers

an aggregation operator which maps a number of non negative grades, G1,...,Gn, to an G1 Gn aggregate grade G. The author studies operators of the form G = logp((p + ... + p )/n), but no axiomatisation is given. Yet this class bears resemblance to generalised scoring rules with p → 0 giving us the minimum grade, p = 1 the average, and p → ∞ the maximum. In a broader picture one could ask if there is a common criterion that motivates the geometric weights used in scoring rules, aggregation operators, exponential discounting of payoffs, the exponential distribution, and others.

Appendix A. Definitions Definition 1. For every number m of candidates, a scoring rule defines a sequence of m m m m real numbers s1 , . . . , sm. A candidate receives a score si for position i in an individual ranking. The sum of scores across all ranking gives the candidate’s total score. The total scores determine the total ranking: candidates with higher total scores are ranked higher, candidates with equal total scores are ranked equally. For example, plurality is the scoring rule with scores (1, 0,..., 0) for each m, while an- tiplurality corresponds to scores (1,..., 1, 0) and Borda – to (m − 1, m − 2,..., 1, 0).

Definition 2. For every number m of candidates, a generalised scoring rule defines t m,k m,k sequences of m real numbers s1 , . . . , sm – one sequence for each tie-breaking round m,k k = 1, . . . , t. In round k, a candidate a receives score si for position i in an individual ranking. The total sum of scores gives a total score sk(a) of candidate a. The total scores determine the total ranking lexicographically: a is ranked higher than b if sk(a) > sk(b) for some round k ≤ t and sl(a) = sl(b) for all l < k. Candidates a and b are equally ranked if sk(a) = sk(b) for all rounds k ≤ t. For example, for each m candidates, generalised plurality has m − 1 rounds with scores k m−k m−k k (1z,...,}| 1{, z0,...,}| 0){ in round k. Generalised antiplurality has (1z,...,}| 1{, 0z,...,}| 0){ .

Appendix B. Proofs

m m Theorem 1. Suppose a scoring rule uses scores s1 , . . . , sm to score m athletes. The scoring m m rule satisfies independence of unanimous losers if and only if s1 > . . . > sm and the scores for k k m m k athletes, s1, . . . , sk, are linearly equivalent to the first k scores for m athletes, s1 , . . . , sk , for all k ≤ m. m m The rule satisfies independence of unanimous winners if and only if s1 > . . . > sm and k k the scores for k athletes, s1, . . . , sk, are linearly equivalent to the last k scores for m athletes, m m sm−k+1, . . . , sm, for all k ≤ m. 14

Proof. The “if” part is straightforward. Let us prove the “only if” part.

For a fixed k ≤ m, consider a profile Pk consisting of just one race, a1 ... ak. By k k independence of unanimous losers, ak must come last, so sk < si for all i < k. Moreover, the

ranking of a1, . . . , ak−1 must be the same as the ranking in the profile Pk−1 with the single

race a1 ... ak−1. By independence of unanimous losers, ak−1 must come last in Pk−1, k k so ak−1 must come second-to-last in Pk, and thus sk−1 < si for all i < k − 1. By repeating k k this argument we establish that s1 > . . . > sk for all k ≤ m. m m Having established that the scores must be strictly decreasing, consider s1 , . . . , sk and k k m m k k k m k m k k s1, . . . , sk. Let α = (s1 − s2 )/(s1 − s2) and β = (s1s2 − s2s1 )/(s1 − s2). Observe that the k k k k scores αs1 + β, . . . , αsk + β are linearly equivalent to s1, . . . , sk, and moreover: m m k m k m m k m k k s1 − s2 k s1s2 − s2s1 s1 s1 − s1 s2 m αs1 + β = k k s1 + k k = k k = s1 , s1 − s2 s1 − s2 s1 − s2 m m k m k m m k m k k s1 − s2 k s1s2 − s2s1 s2 s1 − s2 s2 m αs2 + β = k k s2 + k k = k k = s2 . s1 − s2 s1 − s2 s1 − s2 k m It remains to show that αsi + β = si for i = 3, . . . , k to prove that the scores are linearly k equivalent. For convenience, we write ti = αsi + β for i = 3, . . . , k. m m Suppose for contradiction that ti > si for some i (the case where ti < si is analogous).

Choose integers n2 > 0 and n1 such that: m m m s2 − ti n1 s2 − si (1) m m < < m m . s1 − s2 n2 s1 − s2

Let n = |n1| + n2. Construct a profile with 3n races and m athletes as follows.

If n1 ≤ 0, then in n − n2 races a has position 1 and b has position 2. In n2 races a has

position 1 and b has position i. In n races a has position 2 and b has position i. In −n1 races

a has position i and b has position 2. In n + n1 races a has position i and b has position 1.

If n1 > 0, then in n races a has position 1 and b has position i. In n2 races a has position 2 and b has position i. In n − n2 races a has position 2 and b has position 1. In n2 + n1 races a has position i and b has position 1.

In both cases there are m − k athletes who come last in the order ak+1 ... am in every race, and the other athletes are ranked arbitrarily. Observe than in a profile so constructed athlete a finishes n times in positions 1, 2, i.

Athlete b finishes first n + n1 times, second n − n1 − n2 times, and i-th n + n2 times. The m m m m m m score of a is thus ns1 +ns2 +nsi and the score of b is (n+n1)s1 +(n−n1−n2)s2 +(n+n2)si . m m m m The difference between the scores of a and b is (n2s2 − n2si ) − (n1s1 − n1s2 ), which is positive by formula (1). Thus, a beats b.

Now suppose we drop ak+1, . . . , am from the races. Now the score of a is linearly equivalent m m m m to ns1 + ns2 + nti, and the score of b is linearly equivalent to (n + n1)s1 + (n − n1 − n2)s2 + 15

m m m (n + n2)ti. This gives us a difference of (n2s2 − n2ti) − (n1s1 − n1s2 ) < 0, meaning that dropping the unanimous losers made b overtake a. The argument for independence of unanimous winners is analogous. 

Proposition 1. For any p < ∞, there exists a profile where the overall winner does not come first in any race.

Proof. Suppose p < 1. Consider a profile of n = m − 1 races, m ≥ 3, where athlete a comes second in every race, and has a score of (m − 1)(1 − pm−2). Every other athlete comes first, third, fourth, and so on, exactly once. This gives them a score of 1 − pm−1 + 1 − pm−3 + ... + 1 − 1 = m − 1 − pm−1 − (pm−2 − 1)/(p − 1). We want to show that the difference between the scores of a and every other athlete is positive: pm−2 − 1 (m − 1)(1 − pm−2) > m − 1 − pm−1 − , p − 1 pm−2 − 1 −mpm−2 + pm−2 + pm−1 + > 0, p − 1 mpm−1 − mpm−2 − pm + 1 > 0.

If we take the derivative with respect to p, we get m(m−1)pm−2 −m(m−2)pm−3 −mpm−1 that has the same sign as (m − 1)p − (m − 2) − p2. This is a parabola with vertex at p = (m − 1)/2 and roots at 1, m − 2. Thus for 0 ≤ p < 1, this is a monotonely decreasing function, reaching a minimum as p → 1. At p = 1, m · 1m−1 − m · 1m−2 − 1m + 1 = 0, so for the relevant values of p the difference is positive. Suppose p = 1. Construct the same profile. Athlete a has a score of (m − 1)(m − 2) while the other athletes (m − 1) + (m − 3) + ... + 1 = m − 1 + (m − 3)(m − 2)/2. We want to show that the difference between the scores is positive: (m − 3)(m − 2) (m − 1)(m − 2) > m − 1 + , 2 m2 − 5m + 6 m2 − 3m + 2 > m − 1 + , 2 m2 − 3m > 0.

Which is true for m ≥ 4. Suppose p > 1. Construct the same profile, but with m > p2/(p − 1). Athlete a has a score of (m − 1)pm−2, the other athletes pm−1 + pm−3 + ... + 1 = pm−1 + (pm−2 − 1)/(p − 1). 16

We want to show that the difference is positive: pm−2 − 1 (m − 1)pm−2 > pm−1 + , p − 1 pm−2 − 1 mpm−2 − pm−2 − pm−1 − > 0, p − 1 mpm−1 − mpm−2 − pm + 1 > 0, mpm−2(p − 1) − pm + 1 > 0.

Since we assumed that m > p2/(p − 1):

mpm−2(p − 1) − pm + 1 > pm−2p2 − pm + 1 > 0.

 Theorem 3. Generalised plurality is the only generalised scoring rule satisfying the majority criterion and independence of unanimous winners. Symmetrically, generalised antiplurality is the only generalised scoring rule that satisfies independence of unanimous losers and always ranks the majority loser last. Proof. First observe that the axioms are logically independent. Any geometric scoring rule is a generalised scoring rule that satisfies independence of unanimous winners (Theorem 2), but we will show that only generalised plurality satisfies the majority criterion. Plurality is a generalised scoring rule that satisfies the majority criterion, but not independence of unanimous winners. The Kemeny rule satisfies the majority criterion and independence of unanimous winners, but is not a generalised scoring rule. That generalised plurality satisfies the majority criterion and independence of unanimous winners is straightforward. We shall prove the other direction. Suppose a generalised scoring rule satisfies independence of unanimous winners and the majority criterion. We will show by induction that in the kth round the scores must be k m−k linearly equivalent to (1z,...,}| 1{, 0z,...,}| 0){ . Suppose for all l < k, in the lth round the scores are 1 for the first l positions and 0 elsewhere. In the kth round we are concerned with those candidates that were tied in the first k − 1 rounds, and thus have exactly the same number of first places, second places, through to (k − 1)th places. If m = k we have a perfect tie, and there is nothing more we can do with scoring rules. Assume then that m > k. Since the relevant candidates are tied in the first k − 1 rounds, we can without loss of generality assume that s1 = ... = sk, since the first k − 1 scores will not change the relative scores in any way. We shall first show that sk > si, for all i > k.

Consider a profile consisting of one race, a1 ... am. By repeatedly applying indepen- dence of unanimous winners, it follows that the aggregate ranking must also be a1 ... am.

Since ak must finish ahead of ai for all i > k, it follows that sk ≥ si for all i > k. 17

A round with all scores equal is redundant. Hence, without loss of generality, assume

sk > sj for some j. If m = k + 1, the scores must be linearly equivalent to (1,..., 1, 0), and we are done. Assume then that m > k + 1.

Suppose for contradiction sk = si for some i > k, i 6= k, j. Consider a profile consisting of

three races. In all three races al, for l < k, is ranked in position l. In two races ak is ranked

in position k and ai in position i. In one race ak is ranked in position j and ai in position k.

The difference between the scores of ai and ak is positive, 3sk − 2sk − sj = sk − sj. Thus,

ai beats ak. However, by applying independence of unanimous winners k − 1 times we can

delete a1 through ak−1 without changing the relative ranking of the remaining candidates,

but at that point we run into a contradiction because ak is now the majority winner and

should be ranked first. Hence, sk > si for all i > k.

Next, we will show that all other scores are equal. Suppose for contradiction si > sj for

some i, j > k. Choose an integer n > (sk − si)/(si − sj) > 0. Consider a profile consisting of

2n+1 races. In n+1 races ak has position k and ai has position i. In n races ak has position

j and ai has position k. The candidates a1, . . . , ak−1 hold the first k − 1 positions as before.

The difference between the scores of ai and ak is positive, nsk + (n + 1)si − (n + 1)sk − nsj =

si − sk + n(si − sj) > 0. Again we apply independence of unanimous winners and find that

ai beats the majority winner. It must follow that, si = sj for all i, j > k and the scores are k m−k linearly equivalent to (1z,...,}| 1{, 0z,...,}| 0){ . The argument for generalised antiplurality is analogous. 

Theorem 4. Borda is the unique scoring rule satisfying reversal symmetry and one of independence of unanimous winners or independence of unanimous losers.

Proof. First observe that the axioms are logically independent. Any geometric scoring rule satisfies independence of unanimous losers and independence of unanimous winners by The- orem 2, but we shall see that only Borda satisfies reversal symmetry. Dictatorship satisfies independence and reversal symmetry, but is not a scoring rule. Any self-dual scoring rule satisfies reversal symmetry from the characterisation of Llamazares and Pe˜na(2015). That Borda satisfies independence of unanimous losers follows from Theorem 2. Let us show that the rule satisfies reversal symmetry. For a race, if an athlete gets x points, then for the reversed result of the race the athlete gets m − 1 − x points. Hence, for a profile with n races, if an athlete gets z total points, then for the reversed profile the athlete gets (m − 1)n − z total points. Thus, for a profile, if an athlete is the unique winner and has a higher score than every other athlete, then for the reversed profile this athlete has a lower score than every other athlete. We shall show that it is the only scoring rule which has these properties. 18

We will proceed by induction on the number of athletes. In the base case m = 2, and the only scoring rule which satisfies independence of unanimous losers has scores linearly equivalent to (1, 0), as does Borda. For the inductive case, suppose that Borda is the unique rule satisfying these axioms for m − 1 athletes. By Theorem 1, the scores for m athletes must be linearly equivalent to (m − 1, m − 2,..., 1, y), with y < 1. Suppose for contradiction that y > 0. Consider a profile consisting of 2(m − 1) races. In m − 1 races a1 finishes first and every other aj finishes once at every position except for the

first position. In the other m − 1 races the reverse is true – a1 always finishes last and every other aj finishes once at every position except for the last position. The score of a1 is thus 2 2 (m−1) +(m−1)y. The score of every other aj is m−2+...+1+y+m−1+...+1 = (m−1) +y which is less than the score of a1. For the reversed profile the scores are the same. Hence, a1 wins in both profiles which contradicts reversal symmetry. Suppose for contradiction that y < 0 and m is odd. Consider a profile consisting of m − 1 races. Athlete a1 always finishes in the middle position (m+1)/2, and every other aj finishes 2 once at every position except this middle position. The score of a1 is thus (m − 1) /2. The 2 score of every other aj is m − 1 + ... + 1 + y − (m − 1)/2 = (m − 1) /2 + y which is less

than the score of a1. For the reversed profile the scores are the same, contradicting reversal symmetry. Suppose for contradiction that y < 0 and m is even. Consider a profile consisting of

2(m − 1) races. In m − 1 races a1 finishes in position m/2 and every other aj finishes once at every position except the position m/2. The result of other m − 1 races is the reverse –

a1 always finishes in position (m + 2)/2 and every other aj finishes once at every position 2 except position (m + 2)/2. The score of a1 is thus (m − 1) . The score of every other aj is 2(m − 1 + ... + 1 + y) − m/2 − (m − 2)/2 = (m − 1)2 + 2y which is less than the score of

a1. For the reversed profile the scores are the same, again contradicting reversal symmetry. Hence, y = 0 and we get the Borda rule for m athletes. The proof of the case of independence of unanimous winners is analogous. 

Calculations in Table 4. Here we show how we get the bounds on the level of misery.

Suppose we have n agents, choosing from m films, using a scoring rule with parameter p. For a profile, we call a film t-best if it is among t-best films for all agents. For fixed n, m, p, we want to find the minimal t such that for every profile the chosen film is t-best. Our proof is based on worst-case analysis. The best situation for a non-t-best film is getting (n − 1) first positions and exactly one (t + 1)-th position. Hence, the maximal score of a non-t-best film is: (n − 1)(1 − pm−1) + 1 − pm−t−1. 19

The number of non-t-best films is not more than n(m − t). Thus, the number of t-best films is not less than m − nm + nt. In the worst case, all t-best films fill positions from (t − m + nm − nt + 1) to t. Hence, the average score of a t-best film is not less than: n(1 − pm−t+m−nm+nt−1 + ... + 1 − pm−t) . m − nm + nt We apply the next sufficient condition: the chosen film is guaranteed t-best if the average score of t-best films is more than the maximal score of a non-t-best film: n(1 − pm−t+m−nm+nt−1 + ... + 1 − pm−t) > (n − 1)(1 − pm−1) + 1 − pm−t−1, m − nm + nt npm−t(1 − pm−nm+nt) n − > n − (n − 1)pm−1 − pm−t−1, (m − nm + nt)(1 − p) np(1 − pm−nm+nt) (n − 1)pt + 1 − > 0. (m − nm + nt)(1 − p) We fill the table with minimal t satisfying the inequality above. However, given the quirks of working with integers, the bounds are not necessarily tight. For example, consider the case of n = 2, m = 25, p = 0.7, t = 14 – this fails the inequality. Here, the maximal score of a non-t-best film is 2 − 0.725−1 − 0.725−15 ≈ 1.9716. The number of non-t-best films is not more than 22. The number of t-best films is not less than 3. In the worst case, exactly three t-best films fill positions from 12 to 14 for both agents. Hence, at least one t-best film is ranked 13th or higher by both agents and has a score not less than 2(1 − 0.725−13) ≈ 1.9723. Applying the same arguments, one can prove that the cases of n = 2, m = 25, p = 0.8, t = 15 and n = 2, m = 50, p = 0.9, t = 31 also fail the sufficient condition but are still satisfactory.

References Aggarwal, M. (2015). Compensative weighted averaging aggregation operators. Applied Soft Computing, 28:368–378. Aleskerov, F. T., Chistyakov, V. V., and Kalyagin, V. A. (2010). Social threshold aggrega- tions. Social Choice and Welfare, 35(4):627–646. Amer-Yahia, S., Roy, S. B., Chawlat, A., Das, G., and Yu, C. (2009). Group recommendation: Semantics and efficiency. Proceedings of the VLDB Endowment, 2(1):754–765. Arrow, K. J. (1950). A difficulty in the concept of social welfare. Journal of Political Economy, 58(4):328–346. Bartholdi, J., Tovey, C. A., and Trick, M. A. (1989). Voting schemes for which it can be difficult to tell who won the election. Social Choice and Welfare, 6(2):157–165. Bossert, W. and Suzumura, K. (2018). Positionalist voting rules: A general definition and axiomatic characterizations. Technical report. 20

Brill, M., Laslier, J.-F., and Skowron, P. (2018). Multiwinner approval rules as apportion- ment methods. Journal of Theoretical Politics, 30(3):358–382. Campbell, D. E. and Kelly, J. S. (2002). Impossibility theorems in the Arrovian framework. In Arrow, K. J., Sen, A. K., and Suzumura, K., editors, Handbook of Social Choice and Welfare, volume 1, pages 35–94. North-Holland, Amsterdam. Cervone, D. P., Gehrlein, W. V., and Zwicker, W. S. (2005). Which scoring rule maximizes Condorcet efficiency under IAC? Theory and Decision, 58(2):145–185. Chebotarev, P. Y. and Shamis, E. (1998). Characterizations of scoring methods for preference aggregation. Annals of Operations Research, 80:299–332. Ching, S. (1996). A simple characterization of plurality rule. Journal of Economic Theory, 71(1):298–302. Churilov, L. and Flitman, A. (2006). Towards fair ranking of Olympics achievements: the case of Sydney 2000. Computers & Operations Research, 33(7):2057–2082. de Borda, J. C. (1781). M´emoire sur les Elections´ au Scrutin. Histoire de l’Acad´emieRoyale des Sciences, Paris. Diss, M., Kamwa, E., Moyouwou, I., and Smaoui, H. (2019). Condorcet efficiency of general weighted scoring rules under IAC: indifference and abstention. hal-02196387. Dwork, C., Kumar, R., Naor, M., and Sivakumar, D. (2001). Rank aggregation methods for the web. In Proceedings of the 10th International Conference on World Wide Web, pages 613–622, New York, NY, USA. ACM. Felsenthal, D. S. and Nurmi, H. (2018). Voting Procedures for Electing a Single Candidate: Proving Their (In) Vulnerability to Various Voting Paradoxes. Springer. Fishburn, P. C. (1973). The Theory of Social Choice. Princeton University Press. Fishburn, P. C. (1981). Inverted orders for monotone scoring rules. Discrete Applied Math- ematics, 3(1):27–36. Fraenkel, J. and Grofman, B. (2014). The Borda Count and its real-world alternatives: Comparing scoring rules in Nauru and Slovenia. Australian Journal of Political Science, 49(2):186–205. Freeman, R., Brill, M., and Conitzer, V. (2014). On the axiomatic characterization of runoff voting rules. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, pages 675–681. AAAI Press. Green, R. H., Doyle, J. R., and Cook, W. D. (1996). Preference voting and project ranking using DEA and cross-evaluation. European Journal of Operational Research, 90(3):461– 472. Jameson, A. and Smyth, B. (2007). Recommendation to groups. In Brusilovsky, P., Kobsa, A., and Nejdl, W., editors, The Adaptive Web, pages 596–627. Springer Berlin Heidelberg. 21

Janson, S. (2018). Phragm´en’s and Thiele’s election methods. arXiv preprint arXiv:1611.08826v2. Kamwa, E. (2019). On the likelihood of the Borda effect: The overall probabilities for general weighted scoring rules and scoring runoff rules. Group Decision and Negotiation, 28(3):519–541. Kemeny, J. G. (1959). Mathematics without numbers. Daedalus, 88(4):577–591. Kondratev, A. Y. and Nesterov, A. S. (2019). Measuring majority power and veto power of voting rules. Public Choice. Lackner, M. and Skowron, P. (2019). Consistent approval-based multi-winner rules. arXiv preprint arXiv:1704.02453v6. Lepelley, D. (1992). Une caract´erisationdu vote `ala majorit´esimple. RAIRO-Operations Research, 26(4):361–365. Lepelley, D., Moyouwou, I., and Smaoui, H. (2018). Monotonicity paradoxes in three- candidate elections using scoring elimination rules. Social Choice and Welfare, 50(1):1–33. Lin, S. (2010). Rank aggregation methods. Wiley Interdisciplinary Reviews: Computational Statistics, 2(5):555–570. Llamazares, B. and Pe˜na,T. (2015). Scoring rules and social choice properties: some char- acterizations. Theory and Decision, 78(3):429–450. Masthoff, J. (2004). Group modeling: Selecting a sequence of television items to suit a group of viewers. In Personalized Digital Television, pages 93–141. Springer, Dordrecht. Masthoff, J. (2015). Group recommender systems: Aggregation, satisfaction and group attributes. In Ricci, F., Rokach, L., and Shapira, B., editors, Recommender Systems Handbook, pages 743–776. Springer US, Boston, MA. McCarthy, K., McGinty, L., Smyth, B., and Salam´o,M. (2006). The needs of the many: A case-based group recommender system. In Advances in Case-Based Reasoning, pages 196–210. Springer Berlin Heidelberg. McLean, I. and Urken, A. B. (1995). Classics of Social Choice. The University of Michigan Press, Ann Arbor. Nanson, E. J. (1882). Methods of election. Transactions and Proceedings of the Royal Society of Victoria, 19:197–240. Nash, J. (1950). The bargaining problem. Econometrica, 18(2):155–162. Ozt¨urk, Z. E. (2018). A composition-consistency based characterization of the Plurality rule. Technical report. O’Connor, M., Cosley, D., Konstan, J. A., and Riedl, J. (2001). PolyLens: A recommender system for groups of users. In Prinz, W., Jarke, M., Rogers, Y., Schmidt, K., and Wulf, V., editors, Proceedings of the Seventh European Conference on Computer-Supported Co- operative Work, pages 199–218. Springer, Dordrecht. 22

Reilly, B. (2002). Social choice in the south seas: Electoral innovation and the Borda count in the pacific island countries. International Political Science Review, 23(4):355–372. Richelson, J. T. (1978). A characterization result for the plurality rule. Journal of Economic Theory, 19(2):548–550. Saari, D. G. (1989). A dictionary for voting paradoxes. Journal of Economic Theory, 48(2):443–475. Saari, D. G. and Barney, S. (2003). Consequences of reversing preferences. The Mathematical Intelligencer, 25(4):17–31. Sanver, M. R. (2002). Scoring rules cannot respect majority in choice and elimination simul- taneously. Mathematical Social Sciences, 43(2):151–155. Skowron, P., Faliszewski, P., and Slinko, A. (2019). Axiomatic characterization of committee scoring rules. Journal of Economic Theory, 180:244–273. Smith, J. H. (1973). Aggregation of preferences with variable electorate. Econometrica, 41(6):1027–1041. Thiele, T. N. (1895). Om flerfoldsvalg. Oversigt over det Kongelige Danske Videnskabernes Selskabs Forhandlinger, 1895:415–441. Thomson, W. (2011). Consistency and its converse: an introduction. Review of Economic Design, 15(4):257–291. Tideman, T. N. (1987). Independence of clones as a criterion for voting rules. Social Choice and Welfare, 4(3):185–206. Young, H. P. (1975). Social choice scoring functions. SIAM Journal on Applied Mathematics, 28(4):824–838. Young, H. P. (1988). Condorcet’s theory of voting. The American Political Science Review, 82(4):1231–1244.