Statistical Science 2011, Vol. 26, No. 4, 517–527 DOI: 10.1214/09-STS295 c Institute of Mathematical Statistics, 2011 The 2004 Venezuelan Presidential Recall Referendum: Discrepancies Between Two Exit Polls and Official Results Raquel Prado and Bruno Sans´o Abstract. We present a simulation-based study in which the results of two major exit polls conducted during the recall referendum that took place in Venezuela on August 15, 2004, are compared to the of- ficial results of the Venezuelan National Electoral Council “Consejo Nacional Electoral” (CNE). The two exit polls considered here were conducted independently by S´umate, a nongovernmental organization, and Primero Justicia, a political party. We find significant discrepan- cies between the exit poll data and the official CNE results in about 60% of the voting centers that were sampled in these polls. We show that discrepancies between exit polls and official results are not due to a biased selection of the voting centers or to problems related to the size of the samples taken at each center. We found discrepancies in all the states where the polls were conducted. We do not have enough information on the exit poll data to determine whether the observed discrepancies are the consequence of systematic biases in the selection of the people interviewed by the pollsters around the country. Neither do we have information to study the possibility of a high number of false or nonrespondents. We have limited data suggesting that the dis- crepancies are not due to a drastic change in the voting patterns that occurred after the exit polls were conducted. We notice that the two exit polls were done independently and had few centers in common, yet their overall results were very similar. Key words and phrases: Exit poll data, Venezuelan recall referendum. arXiv:1205.3320v1 [stat.ME] 15 May 2012 1. INTRODUCTION timing and the validity of the process that led to A presidential recall referendum (RR) took place the RR actually taking place. The Organization of in Venezuela on August 15, 2004. A tense political American States (OAS) sent a delegation chaired by debate preceded the RR, the main issue being the its Secretary General to negotiate a solution. The Carter Center, led by President Jimmy Carter him- Raquel Prado is Associate Professor, Department of self, played an important role in getting the govern- Applied Mathematics and Statistics, Baskin School of ment and the opposition to agree on a course of ac- Engineering, University of California, Santa Cruz, 1156 tion. The Consejo Nacional Electoral (CNE) was the High Street, Mail Stop: SOE, Santa Cruz, California official body in charge of the organization of the RR. 95064, USA e-mail: [email protected]. Bruno Sans´o is Professor and Department Chair, Department of Applied Mathematics and Statistics, Baskin School of This is an electronic reprint of the original article Engineering, University of California, Santa Cruz, 1156 published by the Institute of Mathematical Statistics in High Street, Mail Stop: SOE, Santa Cruz, California Statistical Science, 2011, Vol. 26, No. 4, 517–527. This 95064, USA e-mail: [email protected]. reprint differs from the original in pagination and typographic detail. 1 2 R. PRADO AND B. SANSO´ Table 1 possibility of fraud in the recall referendum. Other Official results of the 2004 Venezuelan recall referendum related references include Delfino and Salas (2011) Number votes % of votes and Taylor (2007). In particular, for the S´umate and Primero Justicia exit polls, Hausmann and Rigob´on Registered voters 14,037,900 100% show that there are no significant differences be- Casted votes 9,815,631 69.92% tween the official results for the centers in those Number votes % of casted votes polls and the official results for the overall popula- Yes votes 3,989,008 40.64% tion, thus indicating that the selection of the polled No votes 5,800,629 59.10% centers was not particularly biased. Invalid votes 25,994 0.26% In this study we find significant discrepancies be- tween the S´umate and PJ exit poll results and the official results for the majority of the voting cen- Since the RR was seen by all parties involved as ters. We also find (via a simulation study) that the a pivotal event, several organizations set up schemes discrepancies are neither due to chance nor to the to collect exit poll data. In this work we study data exit polls taking too small a sample for each center. from two exit polls collected independently by S´u- While most of the centers in the exit polls were op- mate, a Venezuelan nongovernmental organization erated with voting machines, some manual centers (NGO) and Primero Justicia, a Venezuelan political were also sampled. We consider this small subgroup party. S´umate (http://sumate.org/index.html) separately, since the analysis in these cases is compli- defines its mission as that of “building democracy.” cated by the presence of invalid votes. Invalid votes It played a leading role in the process of collecting are virtually nonexistent for the automatic centers. the signatures needed to call the RR. Primero Justi- S´umate has produced a report on the entire refe- cia (http://www.primerojusticia.org.ve) is a rel- rendum process (S´umate, 2004), which is available atively young party that campaigned actively to re- from their web page. The Carter Center has produ- call the President. The exit poll samples were col- ced two reports, one on the audit of the RR results lected during the day when the RR took place, at (The Carter Center, 2004) and a final report about a number of voting centers around the country. As observing the RR process (The Carter Center, 2005). seen in Table 1, the official results of the recall (avail- Both reports are available at www.cartercenter.org. able from www.cne.gov.ve) were 40.6% in favor of The exit poll data analyzed here were provided recalling the president (Yes vote) and 59.1% against by S´umate. The data are available from S´umate recalling the president (No vote). There was a per- upon request. The official data on the recall referen- centage of 0.3% of invalid votes. These results are in dum were obtained from the official CNE web site sharp contrast with the exit poll results. According www.cne.gov.ve. to S´umate, the percentage of Yes votes was 60.7% and according to Primero Justicia, the percentage 2. EXIT POLL DATA of Yes votes was 60.5%. The large observed discrepancies between these 2.1 S´umate Exit Poll exit polls and the actual official results immediately triggered discussions among experts and nonexperts The exit poll conducted by S´umate consisted of in Venezuela questioning the validity of the polls a sample of 269 voting centers, out of 8279 total and the official results. One of the arguments raised centers. These were located in 23 of the 24 states against the exit polls was that the selection of the and federal entities. The exception being the State polled centers was biased. Another was that exit of Delta Amacuro. Only one center was polled in polls were conducted only until early in the after- each of the states of Cojedes and Amazonas. In all noon, while many of the voting centers stayed open other states at least two centers were polled. Vot- until late at night. Yet another argument was that ing centers in Venezuelan embassies and consulates the exit polls were biasedly conducted by interview- were not included in the polling. A total of 23,827 ers who were prone to choose pro-Yes voters and people were interviewed in these centers. This corre- that No-voters were less inclined to respond to these sponds to a population of 945,074 voters registered polls. in such polled centers. The exit poll was designed Hausmann and Rigob´on (2004) present a compre- by S´umate and the American polling firm Penn, hensive discussion of several issues regarding the Schoen and Berland Associates (PSB). According EXIT POLLS IN THE VENEZUELAN RECALL REFERENDUM 3 Table 2 reason, only 24 out of the 269 centers chosen to be Example of hourly reporting form for the S´umate exit poll polled by S´umate were polled until 11:00 pm. corresponding to the time 7:00–8:00 am We have access to a database where the samples Age are recorded every two hours, as they were reported <30 30–50 >50 by the pollsters. We observed that the data collec- Gender Target Polled Target Polled Target Polled tion process was, for most centers, very regular. We were able to calculate the overall target for each Male 1 2 1 Female 2 2 1 center and compare that to the effectively observed Total 3 4 2 sample. This is important to establish if a center was strongly above or below target. We notice that be- ing below target does not provide useful information to S´umate, the pollsters were volunteers who were about the nonresponse. Indeed, there could be many trained and supervised by S´umate and PSB for more factors affecting the fact that a pollster collected less than a month. They were instructed to follow a pro- samples than planned. tocol to guarantee that the sample had the least pos- 2.2 Primero Justicia Exit Poll sible bias. In particular, strong emphasis was given to the fact that the pollsters should not be iden- The exit poll conducted by Primero Justicia (PJ) tified as members of S´umate or any other politi- consisted of a sample of 258 voting centers located cal group.