Statistical Robustness of Voting Rules Ronald L. Rivest∗ and Emily Sheny Abstract you get on a rainy day (when only some fraction of the vot- ers show up). (We assume here that the chance of a voter We introduce a notion of “statistical robustness” for voting showing up on a rainy day is independent of his preferences.) rules. We say that a voting rule is statistically robust if the winner for a profile of ballots is most likely to be the winner We show that plurality voting is statistically robust, of any random sample of the profile, for any positive sam- while—perhaps surprisingly—approval voting, STV, and ple size. We show that some voting rules, such as plurality, most other familiar voting rules are not statistically robust. veto, and random ballot, are statistically robust, while others, We consider the property of being statistically robust a such as approval, score voting, Borda, single transferable vote desirable one for a voting rule, and thus consider lack of (STV), Copeland, and Maximin are not statistically robust. such statistical robustness a defect in voting rules. In gen- Furthermore, we show that any positional scoring rule whose eral, we consider a voting rule to be somewhat defective if scoring vector contains at least three different values (i.e., any applying the voting rule to a sample of the ballots may give positional scoring rule other than t-approval for some t) is not misleading guidance regarding the likely winner for the en- statistically robust. tire profile. Keywords: social choice, voting rule, sampling, statistical One reason why statistical robustness may be desirable robustness is for “ballot-auditing” (Lindeman and Stark 2012), which attempts to confirm the result of the election by checking that the winner of a sample is the same as the overall winner. 1 Introduction Similarly, in an AI system that combines the recommen- It is well known that polling a sample of voters before an dations of expert subsystems according to some aggregation election may yield useful information about the likely out- rule, it may be of interest to know whether aggregating the come of the election, if the sample is large enough and the recommendations of a sample of the experts is most likely voters respond honestly. to yield the same result as aggregating the recommendations It is less well known that the effectiveness of a sample in of all experts. In some situations, some experts may have predicting an election outcome also depends on the voting transient faults or be otherwise temporarily unavailable (in a rule (social choice function) used. manner independent of their recommendations) so that only We say a voting rule is “statistically robust” if for any a sample of recommendations is available for aggregation. profile the winner of any random sample of that profile is Since our definition is new, there is little or no directly most likely to be the same as the (most likely) winner for the related previous work. The closest work may be that of complete profile. While the sample result may be “noisy” Walsh and Xia (Walsh and Xia 2011), who study various due to sample variations, if the voting rule is statistically “lot-based” voting rules with respect to their computational robust the most common winner(s) for a sample will be the resistance to strategic voting. In their terminology, a voting same as the winner(s) of the complete profile. rule of the form “Lottery-Then-X” (a.k.a. “LotThenX”) first To coin some amusing terminology, we might say that a takes a random sample of the ballots, and then applies voting statistically robust voting rule is “weather resistant”—you rule X (where X may be plurality, Borda, etc.) to the sam- expect to get the same election outcome if the election day ple. Their work is not concerned, as ours is, with the fidelity weather is sunny (when all voters show up at the polls) as of the sample winner to the winner for the complete profile. ∗ Amar (Amar 1984) proposes actual use of the “random bal- Computer Science and Artificial Intelligence Laboratory, lot” method. Procaccia et al. (Procaccia, Rosenschein, and Massachusetts Institute of Technology, Cambridge, MA 02139 Kaminka 2007) study a related but different notion of “ro- [email protected] bustness” that models the effect of voter errors. yComputer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA 02139 The rest of this paper is organized as follows. Section 2 [email protected] introduces notation and the voting rules we consider. We Copyright c 2012, Association for the Advancement of Artificial define the notion of statistical robustness for a voting rule in Intelligence (www.aaai.org). All rights reserved. Section 3, determine whether several familiar voting rules are statistically robust in Section 4, and close with some dis- eliminated. Each ballot is counted as a vote for its highest- cussion and open questions. ranked alternative that has not yet been eliminated. The winner of the election is the last alternative remaining. 2 Preliminaries • Plurality with runoff : The winner is the winner of the Ballots, Profiles, Alternatives. Assume a profile P = pairwise election between the two alternatives that receive (B1;B2;:::;Bn) containing n ballots will be used to de- the most first-choice votes. A = fA ;A ;:::;A g termine a single winner from a set 1 2 m • Copeland: The winner is an alternative that maximizes m of alternatives. The form of a ballot depends on the vot- the number of alternatives it beats in pairwise elections. ing rule used. We may view a profile as either a sequence or a multiset; it may contain repeated items (identical ballots). • Maximin: The winner is an alternative whose lowest score in any pairwise election against another alternative is the greatest among all the alternatives. Social choice functions. Assume that a voting rule (social choice function) f maps profiles to a single outcome (one Other (non-preferential) voting rules we consider are: of the alternatives): for any profile P , f(P ) produces the • Score voting (also known as range voting): Each allow- winner for the profile P . able ballot type is associated with a vector that specifies We allow f to be randomized, in order for “ties” to be a score for each alternative. The winner is the alternative handled reasonably. Our development could alternatively that maximizes its total score. have allowed f to output the set of tied winners; we prefer allowing randomization, so that f always outputs a single • Approval (Brams and Fishburn 1978; Laslier and Sanver alternative. In our analysis, however, we do consider the set 2010): Each ballot gives a score of 1 or 0 to each alter- ML(f(P )) of most likely winners for a given profile. native. The winner is an alternative whose total score is Thus, we say that A is a “most likely winner” of P if no maximized. other alternative is more likely to be f(P ). There may be • Random ballot (Gibbard 1977) (also known as random several most likely winners of a profile P . For most profiles dictator): A single ballot is selected uniformly at random and most voting rules, however, we expect f to act deter- from the profile, and the alternative named on the selected ministically, so there is a single most likely winner. ballot is the winner of the election. Often the social choice function f will be neutral— symmetric with respect to the alternatives—so that chang- 3 Sampling and Statistical Robustness ing the names of the alternatives won’t change the outcome Sampling. The profile P is the universe from which the distribution of f on any profile. While there is nothing in sample will be drawn. our development that requires that f be neutral, we shall sampling process restrict attention in this paper to neutral social choice func- We define a to be a randomized function G P n tions. Thus, for example, a tie-breaking rule used by f in that takes as input a profile of size and an integer k 1 ≤ k ≤ n S this paper will not depend on the names of the alternatives; parameter ( ) and produces as output a sample P k S it will pick one of the tied alternatives uniformly at random. of of expected size , where is a subset (or sub-multiset) P We do assume that social-choice function f is anony- of . mous—symmetric with respect to voters: reordering the bal- We consider three kinds of sampling: lots of a profile leaves the outcome unchanged. • Sampling without replacement. Here GW OR(P; k) pro- We will consider the following voting rules. (For more duces a set S of size exactly k chosen uniformly without details on voting rules, see (Brams and Fishburn 2002), for replacement from P . example.) • Sampling with replacement. Here GWR(P; k) produces Many of the voting rules are preferential voting rules; that a multiset S of size exactly k chosen uniformly with re- is, each Bi gives a linear order Ai1 Ai2 ::: Aim. placement from P . (In the rest of the paper, we will omit the symbols and just • Binomial sampling. Here G (P; k) produces a sample write Ai1Ai2 :::Aim, for example.) BIN S of expected size k by including each ballot in P in the • A positional scoring rule is defined by a vector ~α = sample S independently with probability p = k=n. hα1; : : : ; αmi; we assume αi ≥ αj for i ≤ j. Thus, the output of the voting rule on a sample might be Alternative i gets α points for every ballot that ranks al- j denoted as f(S), or f(G(P; k)), depending on the situation.