The Politics of Polling

HOUSE OF LORDS Select Committee on Political Polling and Digital Media Report of Session 2017–19 The politics of polling Ordered to be printed 20 March 2018 and published 17 April 2018 Published by the Authority of the House of Lords HL Paper 106 Select Committee on Political Polling and Digital Media The Select Committee on Political Polling and Digital Media was appointed by the House of Lords on 29 June 2017 “to consider the effects of political polling and digital media on politics, and to make recommendations”. Membership The Members of the Select Committee on Political Polling and Digital Media were: Baroness Couttie Baroness Janke Baroness Fall Baroness Jay of Paddington Baroness Ford (until 7 November 2017) Lord Lipsey (Chairman) Lord Foulkes of Cumnock Baroness O’Neill of Bengarve Lord Hayward Lord Rennard Lord Howarth of Newport Lord Smith of Hindhead Declaration of interests See Appendix 1. A full list of Members’ interests can be found in the Register of Lords’ Interests: http://www.parliament.uk/mps-lords-and-offices/standards-and-interests/register-of-lords- interests Publications All publications of the Committee are available at: http://www.parliament.uk/political-polling-digital-media/ Parliament Live Live coverage of debates and public sessions of the Committee’s meetings are available at: http://www.parliamentlive.tv Further information Further information about the House of Lords and its Committees, including guidance to witnesses, details of current inquiries and forthcoming meetings is available at: http://www.parliament.uk/business/lords Committee staff The staff who worked on this Committee were: Beth Hooper (Policy Analyst), Sarah Jones (Clerk, from January 2018), Helena Peacock (Clerk, to December 2017), Viv Roach (Committee Assistant, to December 2017), and Morgan Sim (Committee Assistant, from January 2018). Contact details All correspondence should be addressed to the Select Committee on Political Polling and Digital Media, Committee Office, House of Lords, London SW1A 0PW. Telephone 020 7219 3330. Email [email protected] CONTENTS Page Summary 3 Chapter 1: Introduction 7 Political polling in the United Kingdom 7 The Committee’s remit 7 The Committee’s work 11 Chapter 2: Background 12 What is a poll? 12 Who commissions polls? 14 The accuracy of voting intention polls 15 Are the polls getting worse? 16 Is it getting harder to poll? 20 Persuading the public to take part 21 Decline in the value of socio-economic class as a weighting variable 21 Turnout 22 Financial constraints and the impact of the internet 22 Voter volatility 23 The impact of voting intention polling 24 Impact on voters 24 Impact on the media ‘narrative’ 26 Impact on political parties 27 Conclusion 29 Chapter 3: Media reporting of voting intention polls 31 Concerns about media reporting of polls 31 The ‘horse race’ 31 Lack of reference to important caveats 32 Headlines and margins of error 33 What can polling organisations do? 35 The challenges faced by journalists 36 Views on what could be done to improve media reporting of voting intention polls 38 Giving polls less prominence 38 Training for journalists 39 Chapter 4: The use of policy issues polls 42 Methods used by polling organisations 42 Methodological approaches 43 Case study: IPSO complaint upheld against the Daily Express 44 Case study: Survey of members of the Royal College of Physicians 45 The use of polling results 46 Case study: IPSO complaint upheld against The Sun 47 Case study: Press Complaints Commission complaint resolved with the Daily Mirror 49 Conclusion 49 Chapter 5: Oversight of polling 51 Current oversight of polling 51 British Polling Council 51 Market Research Society 53 Media regulators 54 The overall landscape 56 International comparison: France—Commission des Sondages 56 The case for action on regulation 58 The Electoral Commission 59 A co-ordinated approach 60 Banning of polling 62 Chapter 6: Digital media 65 Introduction 65 Digital media and polling 66 Social media 67 The use of social media data for predicting elections 67 Spread of misinformation on social media 70 Social media and political campaigning 74 Options for tackling the challenges posed by social media 77 Summary of conclusions and recommendations 85 Appendix 1: List of Members and declarations of interest 90 Appendix 2: List of witnesses 92 Appendix 3: Informal briefings 96 Appendix 4: Call for evidence 97 Appendix 5: The polling industry in the UK 99 Appendix 6: British Polling Council: A Journalist’s Guide To Opinion Polls 101 Appendix 7: Letter sent to Liaison Committee 107 Appendix 8: Acronyms and glossary 108 Evidence is published online at https://www.parliament.uk/political-polling- digital-media and available for inspection at the Parliamentary Archives (020 7219 3074). Q in footnotes refers to a question in oral evidence. The politics of pollinG 3 SUMMARY Since the first Gallup polls were run in the United Kingdom in the 1930s, political opinion polls have become an established aspect of British politics. For the media and political parties, political opinion polls are integral to understanding the political mood of the nation. Polling results have also come to dominate election coverage, driving the ups and downs, twist and turns of the ‘narrative’ of each election. The public and others are aware that there is always potential for some inaccuracies in polling. Nonetheless, polls that estimate voting intention continue to serve as prominent and highly influential sources of information during election campaigns. In recent years, however, the polling industry has suffered a number of collective failures. The 2015 General Election was widely considered to be an embarrassment for the polling industry. In the weeks prior to election day much of the focus was on the possibility of a hung Parliament, only for a Conservative majority to emerge. 2015 showed just how influential polling had become, both in the extent to which we have come to rely on poll findings to understand political events, and the considerable impact that polling can have. This realisation prompted serious concerns about both the ability of the polling companies to make accurate and useful estimates of the outcomes of national political events, and around the prominence that polling has come to have. These concerns were only compounded when polling companies were seen to fail again, first in the EU Referendum vote in 2016 and then in the 2017 General Election. Expert analysis has already been undertaken to understand the reasons behind these polling failures, most notably the Report of the Inquiry into the 2015 British general election opinion polls, chaired by Professor Patrick Sturgis. This inquiry and subsequent analysis of the 2016 EU Referendum and the 2017 General Election identified important recommendations on how the industry can continue to innovate and adapt to improve its approach to estimating voting intention. Despite the efforts made to identify and address the methodological challenges associated with recent polling failures, important questions remain. Does the recent poor performance signal a broader trend that polling is getting less accurate? Is polling getting harder to do and, if so, why? Ultimately, does it matter if polling continues to produce inaccurate predictions? It was these questions that we sought to address, and the last of these we considered the most pertinent. Our central concern was that, if it is becoming less likely that polls can provide accurate estimates of the likely election outcomes, then there is a significant risk that future elections will be affected by misleading information, potentially distorting the democratic process. The available data on longer-term trends in polling performance suggest that, currently, it would not be correct to say that we are witnessing a decline in the accuracy of polling. That said, although polling performance has not worsened in a statistically significant way, there is little doubt that confidence in polling has been shaken. We saw evidence of this scepticism in the last General Election. The question ‘can we trust the polls?’ featured prominently in the 2017 election coverage and a number of broadcasters told us that they had deliberately reduced the prominence given to voting intention polls in their election coverage. 4 The politics of pollinG A combination of factors is making it harder to estimate political opinion accurately. We heard that it is getting harder to persuade members of the public to take part in polls and surveys, and that shifts in demographic predictors of the vote, and an increasingly volatile electorate, have all made polling harder to do. This means that polling companies continually need to adapt and innovate. We believe that this can only be done successfully if polling performance is routinely and thoroughly reviewed after each election. The polling industry is facing a number of challenges in polling the modern electorate but, in the overwhelming majority of cases, we recognise that polling companies make every effort to ensure accuracy and that they have no reason to deliberately distort poll findings. These efforts, however, can be undermined by the ways in which the media reports on polling. Media coverage of election campaigns has traditionally involved a considerable focus on polling information, but this coverage is not always an accurate reflection of polling data. Headlines that over-emphasise small, insignificant changes in party fortunes are misleading, but is a practice that remains widely prevalent. We believe that the British Polling Council, in collaboration with other expert groups and regulators, should use its considerable expertise to develop further guidance for the media on the reporting of polls, and work proactively to highlight particularly bad examples of media reporting on polling. There are other types of polls which affect political discourse in the UK, such as those that measure public opinion on political and social issues. We found that some of the key problems we identified for polling, particularly the use of leading questions and misleading presentation of results, were more pronounced for policy issues polls.

Load more