Brexit through the Lens of Data Science
Kenneth Benoit (joint work with Akitaka Matsuo) Department of Methodology LSE
Text mining and data/social science
• Treats text as “data” that informs us about the authors of the text - the text itself is only incidental
• Key is some form of comparison, to gain insights about differences
• Involves the application of statistical models to judge differences using probability statements Analyzing Brexit through Twitter
• EU-funded project
• We collected some 26 million Tweets from January - July 2016
• capture based on #hashtags, @usernames, and search terms Hashtags: Usernames:
#betterdealforbritain @vote_leave #betteroffout @brexitwatch #brexit @eureferendum #euref @ukandeu #eureferendum @notoeu #eusummit @leavehq Search terms #getoutnow @ukineu #leaveeu @leaveeuofficial #no2eu @ukleave_eu #notoeu @strongerin #strongerin @yesforeurope #ukineu @grassroots_out #voteleave @stronger_in #wewantout #yes2eu #yestoeu
brexit Users in data
• 3.6M unique users
• number of tweets:
• average: 7.2
• median: 1 (more than 50% had only one tweet)
• max: 81.1K First application: Predicting Leave v. Remain users
• Method: Naive Bayes classifier
• Data source: combined tweet corpus at user level
• Creating training data
• Select “power-users” (more than 100 tweets in the corpus, 15K users)
• Check the use of pre-determined set of “leave” and “remain” hashtags
• Calculate the difference in the use of leave and remain hashtags. Construct training data from top and bottom 10% of power users Predictive accuracy based on feature types
All Features
Hashtags
Mentions
Hashtags Mentions
No Hashtags Mentions
0% 20% 40% 60% 80% 100% Accuracy
Distribution Bernoulli multinomial
• Use bag of words method (unigrams of #hashtags and @user_mentions, remove features used less than 20 users)
our version Predicting Leave v. Remain
N %
Remain 9,780,223 36.93%
Neutral 7,786,297 29.40%
Leave 8,914,207 33.66% Patterns of posts, Leave v. Remain
1e+06 N 1e+04
Jan Feb Mar Apr May Jun Jul date
side Remain Neutral Leave
our version MP side predictions by party
MP predictions from Tweets
Ulster Unionist Party
UK Independence Party
Social Democratic and Labour Party
Sinn Fein
Scottish National Party side Plaid Cymru Remain Neutral Liberal Democrat Leave
Labour
Independent
Green Party
Democratic Unionist Party
Conservative
0 50 100 150 200 250 1000 A few well- @lsebrexitvote known @simonjhix Twitter users @lseideas
@sarahobolt
side @lseenterprise a Remain @lsesu a Neutral @lsehistory a Leave @kenbenoit @lseekford @lseupr Number of Tweets 10 @lself1202
@lsengers @lserio9762@lseechr1 @lsefton @lsenerazzurra@lsellier@lsekaras
@lseanhubbard@lselawreview@lselectric_dm@lsegreenbelt@lsengler@lseabra@lsenior15@lsesalter@lseror@lsei@lsergy@lserofthyear@lsetlse@lsethy@lselangcentre
0.0 0.4 0.8 1.2 Probability of Remain
#stopthewar #feelthebern #wewantout #takecontrolday #remainers #voteleavetakecontrol Leave and Remain #dontwalkaway #pjnet #loveeuropeleaveeu Leave #specbrexit #believeinbritain #stoptheeu #greatbritain Hashtags #union #euarmy #migrantcrisis #lsebrexitvote #savethenhs #nwo #auspol #wewantourfishback #bluehand #swexit #syria #voteleaveeu
#birmingham #fishingforleave #gogogo #peace #gbpusd #gibraltar #european #incampaign #turnup #jeremycorbyn #notmyvote #freedom #strongerout #trump2016 #press #greece #eusummit #germany #georgeosborne #twibbon #bojo#labourinforbritain #euistheproblem #uktostay #rmt #votetoleave #usd #worldnews #grassrootsout #dexit #out #commonwealth #brexiters #love #remainin #putin #gold #tories #itv #frexit #voteukip #forbes #stock #saferin #german #britin #teamfollowback #davidcameron #usa#beleave #realestate#trading #lbc #isis #irexit #whitegenocide #betterin #ids #nexit #brexitthemovie #registertovote #live #grexit #bnp #farage #forex #rt #tcot #british #racism #fail #gbp #bbc #merkel #juncker #fintech #sky #dodgydave Remain #ukref #money #leaveeu #bbcnews #snp #startup #europeday #pound #euco#fx #ttip #itveuref #follow #marrshow #sovereignty #8217 #ukpolitics #forexnews #gove #brexit #uk #ukip#finance #corbyn #health #forextrading #brexit's #euref#eu #referendum #labour#remainineu #cornwall #tech #london #topstories #bregret#ft #ukineu # #voteremain #eurefresults #sterling #flexcit #remain#r4today #voteleave's #fed #eureferendum #vote #bbcr4today #investment #boe #news #39 #bbcdebate #obama #investing #eupol #leave #bbcqt #bbcdp #jobs #euro #europe #property#brexitrisks #marr #nhs #world #turkey #smallbusiness #yes2eu #cameron #business #bbcsp #tory #regrexit#newsnight #ivoted #immigration #europeanunion #labourin #tomorrowspaperstoday #politics #skypapers #bbcpapers #stocks #c4debate #england #panamapapers #scotland #euro2016 #strongertogether #trump #skynews#pmqs #indyref2 #jobsearch #brussels #peston #reuters #breakingnews #eudebate #migrants #ge2015 Neutral #economy Followership 5.0 davidjo52951945 network mkpdavies analysis, 2tweetaboutit pimpmytweeting
followees’ fight4uk stardust193
nigel_farage 2.5 bluehand007leavethe_eu wantenglandback positions brexitnowwaustraliaunwra6 55massey ukleave_eu elcontador2000beverleytruth end_of_europe gavthebrexitcllrbsilvester patriotic_britpoliticssensestop_the_eu leaveeuofficial euvoteleave23rdtheordinaryman2 (Barberà the_uk_needsyounetworksmanagerjuliet777777proudpatriot101k69atieironwandmikkil bernerlapredhotsquirrel sj_powell scientists4eu dwalingenjasemarkrutter guardianpeston thomasevansukip rcorbettmep agapanthus49 dvatw danhannanmepguidofawkes number10govbbclaurak 2053pamcheekylatte prisonplanet ukip faisalislam alexicon83 breitbartlondon vote_leave raheemkassam skynews paulnuttallukip strongerin 2015) Popularity willowbrookwolflewtonserena5 betteroffoutandrealeadsom montiefrasernelsonbritainelects campbellclaret juliahb1 bbcnewsnight veteransbritaindavidcoburnukip skynewsbreaktelegraphbbcquestiontimebbcr4todayuklabour carolinelucas bristolcomsense steven_woolfe douglascarswelllordashcroft independentpaulwaughchukaumunnaft euromove gjon777 suzanneevans1 bbcnews nataliebendavidallengreeninfactsorghealthierin beeahoney_drteckkhong katehoeympoflynnmep politicshomefinancialtimesgdnpolitics jongaunt johnrentoul britinfluencejolyonmaugham xxplwxx mrharrycole dpjhodges dianejamesmep labourleaveuk__news houseofcommonsmsmithsonpbsuttonnick 0.0 y_euroscepticsholbornlolz liamfoxmp spectatoradamboultonskyshippersunboundbbcnormans strongerinpressjdportes polnyypesets rogerhelmermepeuroguido lbc conhomesophyridgeskysamcoatestimesjimwatersonlabourpressstellacreasyjamesmelvillewdjstrawleftfootfwd students4europe sciencebritainmargotljparker quigleyprichardcalhounlouisemenschtoadmeister europarl_enthegreenpartyharryslaststandlsepoliticsblogbonn1egreerukipnfknanother_europe simplysimontfadcbmep tnewtondunniainmartin1 michaelwhiterhonddabryantanna_soubrystephenkbalbertonardelli a_liberty_rebeljamesdelingpole michaelpdeaconhuffpostukpoliticoeuropefullfact hugodixoncatherinemepsturdyalexremainineuweareeuropeuk brexitthemovie openeuropehuffpostukpolguardianopinionslatukip trevdick vote_leavemediadaily_expressjohnredwoodmike_fabricanttimothy_stanleywallaceme goodwinmj lucycthomasuklabourincer_grant a_brexit_doghelenhimsroundlikejane_collinsmepcolrichardkemp telepolitics chrisshipitvedconwayskynsoamesmpchrisgiles_lsebrexitvoteukandeucer_london env4eur angelneptustar marcherlord1 ajcdeanenadinedorriesmpforbritain david_cameronitvnewsbrexitwatchpeoplesnhsbbcbusinessafp chathamhouse brexpats ruthleaecon stevebakerhwthesuntelegraphnewskayburley marrshow robdothuttonsundersaysbenpbradshawconversationukangiemeader farmers4britainuniforbritainliarpoliticiansbrugesgroup bernardjenkincapx mirrorpoliticspa marycreaghmp otto_english womenforbritaintheredrag bbc5liverachael_swindon38_degreesunitetheunionchunkymarksimon_nixoneuractivwef jude_kd nathangillmepbbcpropagandathescepticislelizbilney georgegalloway bbcrealitycheckred13charlieipsosmoribbckamal opendemocracypeterjukesb_judah racheljoyce truemagic68consforbritain cityam electoralcommukcbitweetsemmareynoldsmpconservativesinjohnharris1969mollymepliberalisland martin_durkineureferendumdailymailukstewart4pboronadhimzahawisocialistvoicereutersuknewsweekthomasbrakeindyvoices alynsmithmep skymurnaghanmetrouktheeconomist georgemagnus1anandmenon1 brexit_news tonyparsonsukowen_patersonmpmrrbournertuknews greghandstimesredboxeuronews leavehqandywigmorewhitewednesdayjamescleverlysputnikint justinegreeningcnbctheipaperprosyn davidickediligenttruthandrew_lilicomrtcharrisreuters nickherbertmpedvaizey timeshigheredthe_tuc pollstationuk businessinsidermarketshelengoodmanmpbitetheballotscotnational davidtcdavies thetimesfreeman_georgehbaldwinmpbbcworldatonesdoughtympfastftmrrae1000 fishingforleave andrewrosindelltimloughtonstandardnewsdrgerardlyonsbrandonlewistelebusinesstheresecoffeysamgyimahfortunemagazinebloombergtvmrjohnnicolson henrysmithmp victorialivedamiancollinsthescotsman john_mills_jml fsb_policythe_iodirishtimesbv globalbritain theopaphitisgeraldhowarthbbcbreakfastben4ipswichibtimesukdwnews stronger_ln annietrev drphillipleempguyoppermandavidgauke akabilky heatstreet murrisonmpmpritchardmp petenorth303kawczynskimp −2.5 nilegardiner
−3 −2 −1 0 1 2 Remain Score
Plot of top 300 hashtags. Cross side edges are highlighted. There are some connections, but the number of edges is smaller than the next figure. Followership 0.4 network analysis, followers 0.3 distribution (Barberà 0.2
2015) Density
0.1
0.0
−2 0 2 Remain Score
Plot of top 300 hashtags. Cross side edges are highlighted. There are some connections, but the number of edges is smaller than the next figure. Sentiment Analysis
• Looks up terms from the Linguistic Inquiry and Word Count, a psychological dictionary
• Contains categories about: • positive and negative emotion • politics • power • quantitative language • tentative language • sadness • future v. past orientation example: “reward” language Reward Language
Remain
Neutral
Leave
0.000 0.002 0.004 0.006 Positive v. Negative Emotion Language
Remain
Neutral
Leave
0.0 0.4 0.8 1.2 Sad Language
Remain
Neutral
Leave
0.0000 0.0005 0.0010 0.0015 0.0020 Future v. Past Language
Remain
Neutral
Leave
−0.4 −0.3 −0.2 −0.1 0.0 Tentative Language
Remain
Neutral
Leave
0.000 0.002 0.004 0.006 Power Language
Remain
Neutral
Leave
0.000 0.005 0.010 Quantitative Language
Remain
Neutral
Leave
0.000 0.002 0.004 0.006 Sentiment Analysis conclusions: Leave was more
• reward-oriented • positive • assertive of power • less quantitative • less tentative • less sadness • future-, versus past-, oriented 2
0
Positive v. Negative Emotion Language v. Negative Positive −2
Jan Feb Mar Apr May Jun Jul Date
side Leave Neutral Remain 0.020
0.015
0.010 Tentative Language Tentative
0.005
Jan Feb Mar Apr May Jun Jul Date
side Leave Neutral Remain Topic models
• Using unsupervised machine learning • detect topics in twitter conversation • topic distributions across sides
• Data • tweets from Leave and Remain accounts identified from NB classification • combine tweets from each account • 700,000 accounts are included
• Method • Structural Topic Model (STM) by Roberts, Stewart, and Tingley (2014) • Estimate models with 10-40 topics (incremented by 5) • covariate in topic prevalence: predicted sides of accounts Donald Trump ● STM Results Islam ● Leave : Debate ● • Selected topics from EU and other "Exit"s ● 40 topics model Leave: Take Control ● Jo Cox and Xenophobia ● Immigration and Identity ● Leave: Messaging ● EU and Sovereignty ● Leave: Grassroots ● Immigration Control ● Vote Mobilisation: Leave ● Brexit and EEA ● Brexit and Market ● Remain: Youth ● Brexit and Economy ● Scotland and Northern Ireland ● Smaller countries "Exit"s ● Remain: Labour and NHS ● Remain : Messaging ● Anger Against Leave Campaign ● Vote Mobilisation: Remain ●
−0.10 −0.05 0.00 0.05 0.10 Effect of Remain STM Results (example of leave topics)
Immigration, Islam Jo Cox and Xenophobia
#wakeupamerica #cameron #refugee #banislam citizen#trump2016 countri #ramadan @sargon_of_akkad #pjnet jo #merkel #britain bilderberg #islam world agenda #usa britain @realalexjones
britain thei victori #austria burn josephreferendum #maga #jocoxmp invas #migrant ar #jocoxsabotag #uk belov rothschild amp
#trump soro
anti vote ago london race #eu christian week #british film voter khan
thei attack
@mailonline blame
follow watch gt support counter ahead nwo mass onli thi #calaisback #australia war #auspol rape menmigrant women racist pro fraudhere' israel dai #ukip cox' #ukip watch ralli stop establish #clinton thi isi ar refuge free kid europ #france watson fals evil fulltruth religion polic #muslim ...... #merkel's merkel sai #orlando speech british jewriot turkei #america #migrants kill elit stop #bluehand #europe noth bbc movi #nyc #deutschland collapswomen paulblack obama #islamic mohamad amp war muslim pleas #britainfirst #islamist worldwid anti rapegovernsadiq divers armi open children lead koran alreadi show they'r dai woman farag @breitbartnews #muslims #pegida #england london reason world happen @prisonplanet violent #euref islam england peopl befor eu #welfare #london #tcot #ccot #christian left politeurop @trobinsonnewera children hate migranttalk #leaveeu#paris media #whitegenocide video #germany #dc #rednationrising death whitecox muslim #obama STM Results (example of remain topics)
Labour, NHS Scotland, Northern Ireland johnson govern ar johnfight protect gove sturgeon workleftcampaign tabl mp referendum ha salmond #euref wing northern english risk nh ireland' agre ttipamp 2nd event ireland sun nicola result wale sai michael #snpin independ bbc #bbcqt support #tories respons campaignirish#scotland part union begun case monei good dure smith britain clean waféin support scotland' corbyn rupertparti snp vote servic tori reason ref europ statement #snp amp ye #labourin climat @jeremycorbyn350m paid govern call niha sai north @carolinelucas highli onlieu indyref patel#marr govt debat uniti case unit council england poor anti fein back poll great
vote pm
reunif border
green scottish media li lie #ttip sinn confer post kingdom rt it' anoth remain remain save place belfast alreadi voter leader chang #ids cut real social clear mornimplic makepartiedinburgh lead pai fraudcultur favour welsh iain hijack glasgow give @thesnp everi thei jeremi peopl elit trust membership #indyref2 job public time uk thi european indi
murdoch tori @alexsalmond welcom #indyref ukip paul voter fake blame #euref man worker auster health week class europpriti democraci care labour#nhs duncan back disabl scotland #greenerin environ @nicolasturgeon eu #voteremain bori #toryelectionfraud STM Results (examples of remain economic topics)
Brexit and Market Brexit and Economy
@business gbpusd lead u.k treasuri latest fall markethigher german central short risk fall cost england global sai back high rate busi investor remain world uncertainti invest @wsj ftse drop investor june vote european referendum rate oecd earli ha@reuterssterlexchang europ ahead biggest crisi biggerboe year #gbp bank ar expatosborn mai high job fx market sai befor bet dow data #stocks effect lose
u. volatildai referendum bond ralli eubritain global doe euro#trading leavaffect low warn show lower imf sector staff potenti rise sinc #auspol#gbpusd uk dealciti ar governor amid trigger price british pound ft growth lead big chief budget carnei economist ha
mai term ceo forex poll hurt long firm fear #fx thi time yearplan ftse london'caus
yen #ausvotes bad shock properti recess chanc todai big uk' put sterl gbp share riskchart billion bank video usd votemorn london eur amp financi industri topcutthreat fear asiabui britain' financi oil yellen impact stock post foreign major
biggest trader british
week #fed post watch economicase trade plung face currenc low damag leader record updatfed ahead gold #gold uk neg impact dollar #pound #forex Table 6: Detected Topics Side Topic Title Remain Brexit ideologues Economic consequence of Brexit Encourage participation Exchange rate Financial risk Globalization and migration Ireland Obama in London Stock market risk StrongerIn Tabloid (Trump, Queen, Jo Cox) Talking about articles on Brexit TopicsTable by 6: Detectedside Topics The city, and big business Side Topic Title Voter registration Remain Brexit ideologues Leave Brexit Movie Economic consequence of Brexit Brexit, UKIP, Leave EU Encourage participation David Cameron and Brussels Exchange rate David Camerons lies Financial risk Debate and discussions Globalization and migration Economic/Financial Sovereignty Ireland Free trade and migrants Obama in London General leave argument Stock market risk Jobs and social security StrongerIn Leave campaign Tabloid (Trump, Queen, Jo Cox) News discussion Talking about articles on Brexit Reasons to Leave EU The city, and big business Take back control Voter registration Undemocratic EU Vote Leave Leave Brexit Movie Neutral Fiscal burden of EU membership Brexit, UKIP, Leave EU Football, nationalism, racism David Cameron and Brussels John Oliver David Camerons lies Opinion polls Debate and discussions Parties and leaders (Scotland) Economic/Financial Sovereignty Uncertainty Free trade and migrants General leave argument Jobs and social security Leave campaign News discussion 18 Reasons to Leave EU Take back control Undemocratic EU Vote Leave Neutral Fiscal burden of EU membership Football, nationalism, racism John Oliver Opinion polls Parties and leaders (Scotland) Uncertainty
18 Exchange rate Networks of topics Stock market risk Obama in London
Financial risk Globalization and migration
Economic consequence of Brexit
The city, and big business
Ireland
Opinion polls
David Cameron and Brussels
News discussion
Talking about articles on Brexit Parties and leaders (Scotland)
Encourage participation Voter registration Brexit ideologues Free trade and migrants Brexit Movie John Oliver Economic/Financial Sovereignty StrongerIn Debate and discussions UncertaintyJobs and social security
Reasons to Leave EU Tabloid (Trump, Queen, Jo Cox) Take back controlUndemocratic EU David Cameron's lies Leave campaign Fiscal burden of EU membership Vote Leave
Brexit, UKIP, GeneralLeave EU leave argument
Football, nationalism, racism
Figure 6: Network of topics. The red circles are Remain topics (judged from the estimated propor- tions of topics, described above), and blue circles are Remain topics. The widths of the lines that connect two topics indicate proximity between two topics measured by the strength of correlations. The distance of two topics in the plot also reflects the proximity.
Table 7: Sides by Source. Twitter YouGov Leave 2515 2030 Remain 2485 2170
19 Summary: Text analysis was used to
• predict the side of the user • determining the most common hashtags by side • using followership networks to estimate “ideology” • measuring sentiment using dictionaries • analyzing the topics that were discussed • mapping connections between topics via networks How? Using software written in R (and C++ and Python)