Revista Internacional de Sociología RIS vol. 74 (3), e043, julio-septiembre, 2016, ISSN-L:0034-9712 doi: http://dx.doi.org/10.3989/ris.2016.74.3.043 SPANISH EXIT POLLS. ENCUESTAS A PIE DE URNA EN Sampling error or nonresponse bias? ESPAÑA. ¿Error muestral o sesgo de no respuesta? JOSE M. PAVÍA ELENA BADAL BELÉN GARCÍA-CÁRCELES University of Valencia University of Valencia University of Valencia [email protected] [email protected] [email protected] Cómo citar este artículo / Citation: Pavía, J. M., E. Copyright: © 2016 CSIC. Este es un artículo de acceso Badal and B. García-Cárceles. 2016. “Spanish exit abierto distribuido bajo los términos de la licencia Creative polls. Sampling error or nonresponse bias?”. Revista Commons Attribution (CC BY) España 3.0. Internacional de Sociología 74(3):e043. doi: http://dx.doi. org/10.3989/ris.2016.74.3.043 Received: 18/11/2014. Accepted: 17/09/2015. Published on line: 23/08/2016 ABSTRACT RESUMEN Countless examples of misleading forecasts on behalf Existe un gran número de ejemplos de predicciones inexac- of both pre-election and exit polls can be found all over tas obtenidas a partir tanto de encuestas pre-electorales the world. Non-representative samples due to differential como de encuestas a pie de urna a lo largo del mundo. La nonresponse have been claimed as being the main rea- presencia de tasas de no-respuesta diferencial entre distin- son for inaccurate exit-poll projections. In real inference tos tipos de electores ha sido la principal razón esgrimida problems, it is seldom possible to compare estimates para justificar las proyecciones erróneas en las encuestas a and true values. Electoral forecasts are an exception. pie de urna. En problemas de inferencia rara vez es posible Comparisons between estimates and final outcomes comparar estimaciones y valores reales. Las predicciones can be carried out once votes have been tallied. In this electorales son una excepción. La comparación entre esti- paper, we examine the raw data collected in seven exit maciones y resultados finales puede realizarse una vez los polls conducted in Spain and test the likelihood that votos han sido contabilizados. En este trabajo, examina- the data collected in each sampled voting location can mos los datos brutos recogidos en siete encuestas a pie de be considered as a random sample of actual results. urna realizadas en España y testamos la hipótesis de que Knowing the answer to this is relevant for both electoral los datos recolectados en cada punto de muestreo puedan analysts and forecasters as, if the hypothesis is reject- ser considerados una muestra aleatoria de los resultados ed, the shortcomings of the collected data would need realmente registrados en el correspondiente colegio elec- amending. Analysts could improve the quality of their toral. Conocer la respuesta a esta pregunta es relevante computations by implementing local correction strate- para analistas y encuestadores electorales, ya que, si se gies. We find strong evidence of nonsampling error in rechaza la hipótesis, las deficiencias de los datos recogidos Spanish exit polls and evidence that the political context deberían ser subsanadas en concordancia. Los analistas matters. Nonresponse bias is larger in polarized elec- podrían mejorar la calidad de sus estimaciones mediante tions and in a climate of fear. la implementación de estrategias de corrección local. En nuestro estudio encontramos una fuerte evidencia de erro- res ajenos al muestreo en las encuestas a pie de urna en España y constatamos la importancia del contexto político. El sesgo de no-respuesta es mayor en elecciones polariza- das y en un clima de violencia o presión. KEYWORDS PALABRAS CLAVE Election night forecasts; measurement error; multi-hyper- Distribución multi-hipergeométrica; Elecciones regionales geometric distribution; nonresponse; Spanish regional españolas; Error de medida; No-respuesta; Predicciones elections. en la noche electoral. 2 . JOSE M. PAVÍA, ELENA BADAL Y BELÉN GARCÍA-CÁRCELES INTRODUCTION potential sources of error, such as interviewer effects (Blom, de Leeuw and Hox 2011), sampling design ef- All around the world, statistical strategies have fects (Pavía and García-Cárceles 2012), noncover- been systematically used for decades to anticipate age by early voting (Mokrzycki, Keeter and Kennedy election results on election nights. In order to inform 2009), measurement errors (Pavía and Larraz 2012), a public eager for immediate data about the results or nonresponse (Groves et al. 2002), which can still of the polls in a timely manner, election prediction seriously compromise their inferences. outcomes have been commissioned for all kinds of electoral races since the 1950s (Mitofsky 1991). Indeed, declining survey response rates are a Depending on the particular political context and the growing concern worldwide (Keeter 2011), which, ac- electoral rules operating in each election, specially- cording to Díaz de Rada (2013), is even threatening suited election night forecasting strategies have been the future of surveys as a tool for gaining proper so- assayed in different countries. There is also a large ciological knowledge, principally when nonresponse tradition of election night forecasting in Spain, with bias is present. Nonresponse bias (i.e., the existence different methods having been tested from both the of systematic differences between respondents and Bayesian (Bernardo 1984, 1997; Bernardo and Girón nonrespondents in the issues of interest) is nowadays 1992) and the frequentist approaches (Pavía, Muñiz considered the main source of error in both pre-elec- and Álvarez 2001; Pavía-Miralles 2005; Pavía, Larraz tion and exit polls (Beltrán and Valdivia 1999; Edison and Montero 2008; Pavía-Miralles and Larraz-Iribas Media Research and Mitofsky International 2005; 2008; Pavía 2010; Sobrino 2012; Martín, Fernández Bautista et al. 2007; Slater and Christensen 2008; and Modroño 2014). Pavía 2010; Pavía and Larraz 2012; Panagopoulus Early returns, quick counts (from a meaningful sam- 2013) and can dramatically damage the quality of ple of representative polling stations) and exit polls are forecasts. In election polls, nonresponse bias arises ordinarily used by the analysts hired by political lead- due to differences in the likelihood of voters of vari- ers and mass media to advance election night final ous parties either providing an answer of their voting results. Early return models and quick-count strate- intentions or partaking in the survey. It may also be a gies rely on actual votes, whereas exit polls —and result of interviewers’ differential propensities to se- entrance polls (Klofstad and Bishin 2012)— rest on lect supporters of different parties (Pavía 2010) due declared votes. Exit-polling strategies, therefore, are to a self-selection mechanism operating in interview- subject, in part, to the miseries of surveys and can suf- ees in response to interviewers features (Haunberger fer some of the shortcomings of pre-election polls, al- 2010), or because some respondents hide their though fortunately not all of them. Unlike a pre-election vote as a consequence of a social desirability bias poll, which asks electors about their intentions several (Krumpal 2013). Furthermore, in exit polls, this could days before the election, an exit poll asks voters about also happen because of different early voting rates an action they have just done. An exit poll is a survey among party supporters (Panagopoulus 2013). of voters interviewed immediately after they have cast There is a growing literature in the US evidencing their ballots at their polling stations. differential nonresponse in US exit polls and proving In an exit-poll dataset, therefore, rather than a that the differences between actual votes and exit- guess of hypothetical actions, the collected data poll projections, measured as “within precinct error” (should) constitute a record of facts just completed. (defined as the difference between the percentage Hence, compared to pre-election (and post-election) margin between the leading candidates in the exit poll polls, exit polls are more reliable and their micro-data and the actual vote), are rising (Merkle and Edelman present some important characteristics that should 2002; Dopp 2006; Frankovic, Paganopoulus and help to improve the accuracy of forecasts. In particu- Shapiro 2009). In this paper, we assess the nonre- lar, with exit-poll data (i) we do not need to identify sponse bias hypothesis in the context of Spanish exit which respondents will actually vote (Selb et al. 2013); polls. To do this, we analyze the relevant micro-data (ii) we do not need to ascertain how undecided voters of seven exit polls conducted in Spain during the last will make their final decisions (Orriols and Martínez ten years in five Spanish regions. In particular, we 2014); (iii) we do not need to worry about voters who study both graphically and statistically (by hypothesis decide to change their mind in the last minute and, testing) whether the vote responses collected in each besides; (iv) we do not need to be so concerned of the voting places can be thought of as a random about coverage issues (Haan, Ongena and Aarts sample of the actual vote distribution recorded in the 2014) or (v) about the fact that different people have polling location. Our hypothesis is that, in general, different probabilities of being reached during the poll the samples cannot be observed as random samples fieldwork (Díaz de Rada 2014). Rather, an exit poll is and that, as we will argue, when the random sample specifically addressed to the voting population and, null hypothesis is rejected the (alternative) most plau- what is more, can collect very large samples in a very sible (albeit not the only) explanation is nonresponse cost-effective manner. However, in the same way as bias caused by differential nonresponse. Knowing the other surveys, exit polls are still exposed to powerful answer to this is relevant for both electoral analysts RIS [online] 2016, 74 (3), e043. REVISTA INTERNACIONAL DE SOCIOLOGÍA. ISSN-L: 0034-9712 doi: http://dx.doi.org/10.3989/ris.2016.74.3.043 SPANISH EXIT POLLS.