Brexit Befuddles British Pollsters

Issue: Brexit

Short Article: Brexit Befuddles British Pollsters

By: Andy McSmith

Pub. Date: August 15, 2016 Access Date: October 2, 2021 DOI: 10.1177/237455680216.n6 Source URL: http://businessresearcher.sagepub.com/sbr-1775-100664-2746302/20160815/short-article-brexit-befuddles-british-pollsters ©2021 SAGE Publishing, Inc. All Rights Reserved. ©2021 SAGE Publishing, Inc. All Rights Reserved. Are telephone or online surveys more accurate? Executive Summary

British polling firms are picking through the rubble of their reputations for clues on why most mistakenly forecast a win for the Remain camp. Full Article

The embarrassment that followed the United Kingdom’s vote for Brexit was not confined to those who had campaigned to remain in the European Union: The pollsters did not cover themselves in glory, either. The betting shops and online betting companies, who do a brisk trade in the U.K. taking wagers on elections and other political events, also took a hit because, after reading the polls, they had offered generous odds to anyone wanting to place a late bet on Brexit winning. Two out of the 10 main polling companies called it right, even if their predictions were off target. The final internet polls conducted by Opinium and TNS both predicted Leave to win 51 percent, with Remain at 49 percent. Since the actual result was Leave 51.9 percent, Remain 48.1 percent, those polls were within the margin of error of 2 percentage points. YouGov conducted its last poll a few days earlier and also called the result roughly right. Meanwhile, one pollster, Populus, gave Remain a 10-percentage-point lead; another, ComRes, put Remain 8 points ahead. Every poll conducted in the last few days underestimated the Leave vote. Averaging out the result to produce a “poll of polls” put Remain at 52 percent and Leave at 48 percent. The numbers, oddly, were right, but they were the wrong way round. The reason polling companies do political polls is not because they are particularly lucrative, but because of the free publicity they attract, which helps establish the pollsters’ brands. However, they need to be able to reassure their commercial clients that they can measure public reactions accurately, so it is not good for business when they very publicly get it wrong. In all, it was an embarrassing reminder of how, in the run-up to the U.K.’s 2015 general election, the polls consistently underestimated the Conservative vote and overestimated Labour’s. One argument during the campaign was over the relative reliability of telephone versus internet surveys. In the final slew of polls, those conducted by telephone gave Remain a 6-point lead on average, while online polls put Remain a mere 2 points ahead. Both approaches have drawbacks. Online polls are weak because they have difficulty drawing in people who are too uninterested in politics to take part, and affluent, socially liberal professionals who don’t have the time to spare. The weakness of telephone polling is the low rate of replies: reaching 2,000 respondents can require dialing 28,000 randomly generated numbers. There is also the problem of “social satisficing,” which makes people inclined to express the opinion they believe to be socially acceptable, rather than the one they actually hold, when talking to someone on the phone. Pollsters believe this is less of a problem in online surveys, which are more anonymous. One of the U.K.’s best-known political pollsters was Peter Kellner, a former journalist whose wife, Cathy Ashton, was the EU’s first foreign minister. Having recently retired, he was able to give a detached view. He acknowledged that “satisficing” might tempt people who proposed to vote Leave to tell telephone pollsters that they were supporting Remain, because that was the “establishment” view, but he said that telephone polls were nonetheless more reliable than online polls. 1 This was also the view reached in a paper published in March by two analysts from Populus. 2 Kellner had several reasons for drawing this conclusion, not least that in the United States, telephone polls had measured Donald J. Trump’s support in Republican presidential primaries more accurately than online polls. 3 This led Kellner to the embarrassing prediction that Remain would win by 8 points. The pollsters have barely begun the painful process of working out why they got it so wrong. One obvious trap was that the turnout, at 72 percent, was higher than for almost any election since 1992. In adjusting their samples, pollsters always allow for the fact that not everyone votes. Thus, in the last week, the fieldwork by the ORB International polling firm and TNS produced identical results – Remain 49, Leave 51. The previous week, TNS had eliminated all but those certain to vote, giving Remain an 8-point lead. In the last week, TNS changed course and stayed with its original figures, to its great relief. ORB stuck to the policy of eliminating those uncertain to vote and – like Kellner –forecast an 8-point win for Remain. After the result, Andrew Hawkins, the chairman of the ComRes polling firm – which had had a good run in recent elections but got the referendum badly wrong – issued a statement in which he suggested that two factors had led their pollsters astray. 4 One was the pollsters’ standard excuse that people changed their minds at the last minute. The more interesting proposition was that pollsters may need to take more account of regional disparities. “We will need to find a way to manage this new dynamic,” Hawkins wrote.

The evidence can be founded in the voter turnout data released by the U.K.’s Electoral Commission. Turnout was below 70 percent in each of the three areas – London, Scotland and Northern Ireland – that voted Remain, and above 70 percent in every other region except for the northeast of England, where it was 69.3 percent. 5 By contrast, in the 2015 general election, Scotland was the only region of the U.K. where turnout was above 70 percent. 6 In the East Midlands, to take one example, turnout was 63.3 percent in 2015 and 74.2 percent a year later, and “Leave” won by 1.5 million votes to 1 million, In all, 3 million people voted in the Brexit referendum who had not voted in the previous year’s general election. The statistical evidence is that the Brexit campaign won by turning out people in the poorer parts of the country, who had perhaps never voted before. It was a localized phenomenon that the pollsters simply failed to see. About the Author

Andy McSmith is a veteran British journalist who has worked for many years as a top political correspondent and political editor for major newspapers in the United Kingdom, including The Independent, The Observer and the Daily Telegraph. He has written books about Great Britain in the Margaret Thatcher era, the British Labour Party and its leaders, and a history of Soviet artists during Stalin’s reign. Notes

[1] Peter Kellner, “EU referendum: ‘remain’ on course for clear victory,” The Politics Counter, May 20, 2016, http://tinyurl.com/zbk48qc. Andrew Hawkins: Fellow pollsters must pay more attention to regional demographics. [2] Matt Singh and James Kanagasooriam, “Polls apart: An investigation into the differences between phone and online polling for the UK’s EU membership referendum,” Populus, March 2016, http://tinyurl.com/jynbect. [3] Nate Cohn, “Is Traditional Polling Underselling Donald Trump’s True Strength,” The New York Times, May 17, 2016, http://tinyurl.com/hndhrao. [4] Andrew Hawkins, “Statement: EU Referendum Polls,” ComRes, June 24, 2016, http://tinyurl.com/hd8ty3c. [5] “EU referendum results,” The Electoral Commission, undated, http://tinyurl.com/zthfzg2. [6] “General Election 2015,” briefing paper, House of Commons Library, July 28, 2015, http://tinyurl.com/pbzb2ud.

Page 3 of 3 Short Article: Brexit Befuddles British Pollsters SAGE Business Researcher