Psephological Fallacies of Public Opinion Polling
Total Page:16
File Type:pdf, Size:1020Kb
SPECIAL ARTICLE Psephological Fallacies of Public Opinion Polling Praveen Rai Opinion polls in India capture electoral snapshots in time he terms “survey” and “opinion poll” in India would that divulge information on political participation, have remained a professional jargon of market research industry, had it not been used for predicting election ideological orientation of voters and belief in core T outcomes. The green shoots of opinion polls to study Indian democratic values. The survey data provides for crucial national elections emerged in the 1950s, but it caught the ima- social science insights, validation of theoretical research gination of the people and became clichéd in the closing dec- and academic knowledge production. Although the ade of the 20th century. The popularity of election surveys stems from the political socialisation and crystal ball gazing media’s obsession with political forecasting has shifted curiosity of Indians to foresee the outcomes of hustings before to electoral prophecy, psephology continues to provide the pronouncement of formal results. The electoral inquisitive- the best telescopic view of elections based on the ness of the stakeholders created a large canvas of opportunity feedback of citizens. The ascertainment of subaltern for opinion-polling industry and scope for scientifi c forecasting of Indian election competitions. The proliferation of electronic opinion by surveys not only broadens the contours of media and the rapid monetisation in the 1990s provided mo- understanding electoral democracy, but also provides mentum to polling agencies to venture into opinion polling on an empirical alternative to the elitist viewpoint of national electoral politics and state election contests. The opin- competitive politics in India. ion polls captured panoramic snapshots and divulged the socio- demographic characteristics of Indian voters and their nuanced voting preferences, as well as reasonably accurate vote esti- mates of political parties for predicting elections. The fi xation for survey-based election prediction turned a host of political scientists and television anchors in India into psephologists. The media election soothsaying became so defi nitive and encap- sulating that it overshadowed the announcements of election re- sults by the Election Commission of India (ECI) to a mere formali- ty of medal distribution ceremony. The success of mediatised election prophecy was short-lived, however, as the erroneous pre- diction of the national election results in 2004 led to widespread public criticisms and calls for a blanket ban of pre-poll, exit poll and post-poll election surveys during elections in India.1 The media-opinion polling industry, facing an existential threat, resorted to course correction, but the election polling ecosystem turned from bad to worse between 2005 and 2013, as political parties were caught manipulating in-house elec- tion survey data for mobilisation of the electorate. This marked a tectonic shift in purpose, as initially political outfi ts commis- sioned election surveys to gauge the mood of voters, collect grassroots feedback for selection of “winnable” candidates and formulation of election strategy and manifesto. Between 2014 and 2019, the correct election forecast of state elections by opinion polls led to rebuilding the confi dence of people in quantitative analysis of election and balloting. The polling in- dustry, with increased accuracy in election forecasting, faced a crucial litmus test in general elections 2014, as the incumbent Congress-led United Progressive Alliance (UPA) was up against Praveen Rai ([email protected]) is a political analyst at the Centre for a resurgent and combative Bharatiya Janata Party (BJP)–led the Study of Developing Societies, Delhi. National Democratic Alliance (NDA). The election prediction of Economic & Political Weekly EPW july 10, 2021 vol lVi no 28 51 SPECIAL ARTICLE most market research agencies correctly fathomed the direc- attitudes. A tension always existed between survey research- tion of the 2014 mandate, but none of them (except one poll) ers who produced pre-election opinion polls and consumers could forecast a BJP majority of seats for the fi rst time in the who watched or read poll stories during election campaigns. lower house of Parliament. The Lok Sabha elections in 2019 The pollsters emphasised that their results are only an election once again posed a major challenge for polling agencies, as “snapshot in time” and not a “political forecast.” While some most Indian and foreign media could not fi nd the prevalence pollsters dismiss the notion of treating pre-election polls pri- of the “Modi wave” and predicted with aplomb that it will be a marily as a forecasting instrument, many in the fi eld are happy waveless national husting. The exit poll fi ndings were diamet- to treat the apparent accuracy of polling near elections as a rically opposite of the media narrative of the fading saffron sign of its overall health. It became implicit in the scoring of wave, as it revealed that the BJP-led right wing alliance would poll accuracy, in Crespi’s (1988) resolution long ago, that rul- do an impressive electoral rebound with a bigger mandate. ing out the forecasting value conducted “immediately before The political forecast by pollsters proved correct, but barring an election” is to “impugn meaningfulness of all polls.” If polls two polling agencies, none could prophesise that the BJP would cannot achieve such predictability, why should we accept any win more than 300 out of the 543 Lok Sabha seats. The predic- poll results as meaning relevant to real life (Crespi 1988; Blu- tion of the people’s mandate by election surveys was in the menthal 2014)? It thus becomes contextual to compile and right direction, but most of them once again failed in correct engage in a comparative analysis of poll predictions with the assessment of the magnitude of the BJP’s political triumph. national elections results to ascertain the accuracy rates of Thus, it becomes important to revisit and review the opinion public opinion polling in India. polls for quantitative analysis of election and balloting from a The general elections in 1998 marked the mushrooming of holistic perspective to fathom their intrinsic limitations, situa- opinion polls to read the minds of voters and the beginning of tional challenges and statistical accuracy errors in predicting a media rat race for seat predictions and eureka moments. The election outcomes. This article comprises of three major parts. majority of election polls predicted the seat tally of the BJP The fi rst collates the accuracy levels of public opinion polls by allies quite closely, except India Today–CSDS (Centre for the media conglomerates of national elections between 1998 and Study of Developing Societies), which underestimated the vic- 2019 in India. A comparative analysis of seat prediction based tory margin of the saffron party and its political partners by a on election surveys with national election verdicts will not wide margin, but surprisingly got it right for the losing Con- only help in foregrounding the precision levels achieved by the gress alliance. The explanation for some discrepancies bet- polling industry in the last two decades, but also map the ween the predicted and actual seats can perhaps be on the gap degree of standard deviations. The archiving will provide the between the date of the election survey and the dates of the growth continuum of election opinion polls and decipher the actual voting. The design of the opinion poll was primarily to changes in psephological trajectory and the methods for sum- ascertain an all-India prediction of seats, and though the CSDS mating public opinion. The second part theoretically fathoms method involved predicting seats for all states, the sample size the biases and fallacies of psephology and the insurmountable at the state level was too small to offer any measure of confi - challenges faced by the polling organisations in conducting dence in the predictions (Karandikar et al 2002). acc urate election surveys. It discusses the inherent errors and The next big test for election forecasting was the general bias in survey research and their impact on the accuracy quo- elections in 1999, which again proved to be a red-letter day for tient of election studies in the multiparty structure of Indian the polling industry. The polls correctly predicted a victory of competitive politics. The fi nal part deconstructs the funda- the BJP alliance, with a slight overestimation of seats for the mental fl aws in opinion polls, scrutinises the reasons for erro- Table 1: General Elections 1998: Almost winning political com- neous poll predictions and probes the media sensibility in dis- Accurate Prediction bine. The election pre- Seat Forecast BJP Allies Congress Allies Others tinguishing between empirical and anecdotal evidence while DRS-Times of India* 249 155 139 dictions based on sur- reporting election data analytics. It will delineate the psepho- Outlook-C Nielsen 238 149 156 veys done by various logical advancement and traction of public opinion polls along India Today-CSDS 214 164 165 media houses during with transmutation of purpose from understanding elections Frontline-CMS** 225–235 145–155 152–182 the general elections to a media utility for political predictions. Actual result 252 166 119 in 1998 and 1999 were Seat predictions based on opinion poll conducted quite accurate and al- before the elections. Accuracy of National Election