THE FLEETING IMPACT OF OUTRAGE – AFP TORONTO ‘17
Session Overview • Welcome • Introduction • Opinion Mining • Case Studies • Q & A
What is ‘Opinion Mining’? “Opinion mining (also known as sentiment analysis) refers to the use of natural language processing, text analysis and computational linguistics to identify and extract subjective information…
Generally speaking, opinion data analysis aims to determine the attitude of a speaker or a writer with respect to some topic…”
• https://en.wikipedia.org/wiki/Sentiment_analysis CASE STUDY
Brexit WHY ACCURATE OPINION DATA MATTERED PRESENT
What the pundits said…
48% Leave, 52% Remain
…if you gave me £100 and offered Natalie Bennett, British Politician me the chance to bet on it, I think I’d say 52.5 for Remain
EU referendum polls: Final ComRes poll shows significant lead for Remain
42% Leave, 58% Remain
46% Leave, 53% Remain http://www.dailymail.co.uk/news/article-3651930/David-Beckham-backs-REMAIN-just-two-days-referendum-urges-voters-staying-children.html What Social Media said
61% 59% 57% 53% 47% 43% 41% 39% LEAVE REMAIN 56.9% 43.1%
20 June 21 June 22 June 23 June Leave Remain Research Methodology • Tracked 500,000 online conversations in the week leading up to the vote. • Identified the city in which the author was located through geo-location algorithms. • Crowd-verified a sample of 10,000 mentions. • Delivered daily sentiment analysis with a 95% confidence interval and a 2.5% margin of error. Brexit prediction by regions
Top cities by volume Top outlying regions
London REMAIN Sheffield LEAVE
Manchester REMAIN Newcastle upon Tyne LEAVE
Edinburgh REMAIN Hull LEAVE
Bristol REMAIN Bournemouth LEAVE
Glasgow REMAIN Coventry LEAVE
Sheffield LEAVE Southampton LEAVE
Cardiff REMAIN Leicester LEAVE
Liverpool REMAIN Chester LEAVE
Birmingham REMAIN Portsmouth LEAVE
CASE STUDY
US Elections 2016 HOW SOCIAL MEDIA DATA TRUMPED THE POLLSTERS DISTRIBUTE
What everybody else said…
What a Clinton landslide would look like http://fivethirtyeight.com/features/what-a-clinton-landslide-would-look-like/
Clinton leads Trump by 5 points, swing states tighten: Reuters/Ipsos http://www.reuters.com/article/us-usa-election-poll-idUSKBN12Z2TX
Clinton leads by 5 heading in to final weeks http://edition.cnn.com/2016/10/24/politics/hillary-clinton-donald-trump-presidential-polls/
Trump won’t win. In fact, the US could be on the brink of a liberal renaissance https://www.theguardian.com/commentisfree/2016/jun/11/trump-cant-win-election-america- political-earthquake Pro-Trump or anti-Clinton unique authors as a percentage of all candidate sentiment
100%
75%
50%
25%
0%
Oct 1 - Oct 23 Oct 24 - Oct 30 Oct 31 - Nov 6 Nov 7 Weighted average Opinion Data accuracy outperforms polls
BBC FIVETHIRTYEIGHT BRANDSEYE
Colorado CLINTON CLINTON TRUMP
Florida TRUMP CLINTON TRUMP
Iowa TRUMP TRUMP TRUMP
Michigan CLINTON CLINTON TRUMP
Nevada TRUMP CLINTON CLINTON
New Hampshire CLINTON CLINTON CLINTON
North Carolina TRUMP CLINTON TRUMP
Ohio TRUMP TRUMP TRUMP
Pennsylvania CLINTON CLINTON TRUMP
Virginia CLINTON CLINTON TRUMP
Wisconsin CLINTON CLINTON TRUMP
7 6 9 out of 11 • Tracked 37 million mentions Research Methodology • Geo-location algorithms identified the state in which the author was located. • A random sample of over 200,000 mentions from key battleground states were sent to our crowd for verification. • They delivered daily net sentiment towards Trump and Clinton with a 95% confidence interval and a 3.5% margin of error. AI is not (yet) the answer
SemEval-2017 Task 4 Sentiment Analysis in Twitter
RANK AI SYSTEM ACCURACY 1 NNEMBs 66.4% Artificial Intelligence 2 LIA 66.1% cannot understand the 3 BB_twtr 65.8% complexity and nuance 4 Senti17 65.2% of language in the world 5 DataStories 65.1% of social media.
Semantic Evaluation 2017 Competition: Qatar Computing Research Institute, SemEval 2017, Task 4, Subtask A. http://alt.qcri.org/semeval2017/task4/index.php?id=results • A mix of machine learning and crowd-sourced
verification to achieve unparalleled accuracy
When machines are through opinion data. For every term or topic combined with the tracked, a statistically significant sample of the power of humans, mentions are processed through human crowds, accuracy can be to understand the nuance of conversation and to achieved at 95%+ accurately identify both the topics driving confidence levels conversation and the sentiment of the authors towards those topics.
• With accurate opinion data strategic and
operational decision-makers in near real-time
have an accurate gauge on what the public is
thinking and feeling.
THE SYRIAN REFUGEE CRISIS: JAN 2015 – SEP 2016 METHODOLOGY
Mentions from 1 Jan 2015 – 30 Sep 2016
5 million mentions about the Syrian refugee crisis
23 k mentions human-verified by the BrandsEye crowd
0.7 margin of error with a 99% confidence level CROWD VERIFIED DATA
The crowd verified mentions from over 20 different languages • Including English, Spanish, Turkish, French and Arabic The crowd also processed data from over 25 different countries ARE PEOPLE PAYING ATTENTION? OVERVIEW On a daily basis, an average of 95 156 people are talking about the Syrian refugee crisis. An average of 201 456 posts on social media about the Syrian refugee crisis daily
14 000 10 316 11 513 11 445 12 000 11 721 9 737 12 380 11 444 10 000 11 240 8 026 10 687 9 848 8 422 6 372 8 000 6 561 6 092 5 632 8 551 5 136 7 363 6 000 6 730 5 353 5 571 6 114 4 000 5 203
2 000
- 12 AM 01 AM 02 AM 03 AM 04 AM 05 AM 06 AM 07 AM 08 AM 09 AM 10 AM 11 AM 12 PM 01 PM 02 PM 03 PM 04 PM 05 PM 06 PM 07 PM 08 PM 09 PM 10 PM 11 PM COUNTRY BREAKDOWN Conversation from 245 countries, regions and islands across the world! UK: 12 000 / day Germany: 2 600 / day
Canada: 3 100 / day
USA: 40 000 / day Syria: 2 400 / day
Turkey: 3 200 / day India: 2 400 / day SENTIMENT BY COUNTRY
Most Negative Most Positive
Iceland 25.2% Lebanon 15.8%
Unites States of America 24.7% Canada 14.1%
Australia 21.5% Ukraine 13.9%
South Africa 20.3% Ireland 13.5%
Turkey 20.3% Germany 13.0%
United Kingdom 20.1% United Kingdom 12.6%
Brazil 20.1% Norway 12.4% MOST ACTIVE AUDIENCES BY TWITTER BASE
Looking at amount of unique authors divided by total registered users on Twitter per country
0.45%
0.40%
0.35%
0.30%
0.25%
0.20%
0.15%
0.10%
0.05%
0.00% Canada Malaysia Australia United United Italy Germany Nigeria Turkey South Kingdom States Africa CONVERSATION WITHIN SYRIA CONVERSATION ABOUT ALEPPO killed civilians people airstrikes injured regime #assad children russian #holocaustaleppo People are talking! …but are they contributing? WHAT ABOUT DONATIONS?
Looking at the comparison of mentions to donation, the story looks dismal…
450,000 Alan Kurdi picture spreads global outrage 400,000
350,000
300,000
250,000
200,000
150,000
100,000
50,000
0 6/1/2015 7/1/2015 8/1/2015 9/1/2015 10/1/2015 11/1/2015 12/1/2015 People donating All people talking DONATIONS – THE CONVERSION RATE
The death of Alan Kurdi caused a massive increase in people donating towards the Syrian Refugee cause. In the months leading up to this terrible tragedy, 1.2% of all people talking about it the Syrian Refugee crisis, made donations. In September, this increased to 11.5%. The months after, 2.9% of people talking about it, made donations. Overall, this is still only 2.7% of people donating to the cause.
450,000 Alan Kurdi picture spreads global outrage 8000
400,000 7000
350,000 6000 300,000 5000 250,000 4000 200,000 3000 150,000 2000 100,000
50,000 1000
0 0 6/1/2015 7/1/2015 8/1/2015 9/1/2015 10/1/2015 11/1/2015 12/1/2015 All people talking People donating *People donating plotted on secondary axis DONATIONS – THE INSIGHT OF CORRELATION
Correlation (definition): noun a mutual relationship or connection between two or more things. There is an ‘R’ of 0.8 – a strong correlation between the volume of conversation about Syrian Refugees and donations made. WHAT DOES THIS MEAN? The450,000 more people are talking about it, the more donations! 8000
400,000 7000 350,000 6000 300,000 5000 250,000 4000 200,000 3000 150,000 2000 100,000
50,000 1000
0 0 6/1/2015 7/1/2015 8/1/2015 9/1/2015 10/1/2015 11/1/2015 12/1/2015 All people talking People donating How do we change this? Campaigns analysis WHAT ARE THE MOST ENGAGING STORIES
MOST ENGAGING POSTS Stories that mostly revolve around children and the human-centred story. The top 135 most engaging mentions received 12% of the total engagement around the Syrian refugee crisis. CAMPAIGN POSTS
Campaigns accounted for 1% of the total engagement
MIXTURE OF POSTS
A mixture of news and human-interest stories and campaigns ENGAGEMENT*
*DEFINED BY THE NUMBER OF SHARES OR COMMENTS POSTS RECIEVE WHAT WERE THE MOST ENGAGING STORIES?
50% of the most engaged with stories referenced children
BBC reporter based in Beirut 10.7k followers 7 y/o Syrian girl tweeting about her experiences of living in British news network Aleppo – 71.1k Followers 1.22M followers WHAT WERE THE MOST ENGAGING STORIES? The human story prevails… THE MOST ENGAGING CALL TO DONATE
GIGI HADID – American fashion model and TV personality
1 655 retweets 3m people potentially saw this HOW DID DONATE CAMPAIGNS FARE? This was only the 119th most engaging post
The post was the most engaged with campaign post – football as a global sport appears to be key here 7.5m people potentially saw this
Similarly this story generated 18k social mentions, with 141m people potentially seeing this MOST ENGAGING AID AGENCY CONTENT AID AGENCY CAMPAIGNS – Content Analysis
Aid agency posts tend to highlight certain instances of the situation in Syria or the refugee crisis. Most do not contain a call to action. Other “campaigns” have been driven by multiple sources THE POWER OF A NAME
“We’re often led to believe migration is a drain on the country’s resources but Steve Jobs was the son of a Syrian migrant. Apple is the world’s most profitable company, it pays over $7bn a year in taxes – and it only exists because they allowed in a young man from Homs.” - BANKSY THE POWER OF STORY
This story of a Syrian man selling pens in Lebanon, while carrying his sleeping daughter spurred ordinary citizens to act.
Within 24 hours, $50 000 had been raised from the crowd- funding platform IndieGogo.
How can we leverage public sentiment for change? The stories of Alan Kurdi and Omran Daqneesh represented a pivotal turning point in how the world saw the Syrian refugee crisis A(Y)LAN KURDI BEFORE & AFTER - ALAN KURDI VOLUME OF CONVERSATION ON SOCIAL MEDIA
300,000 Paris attacks
250,000 Alan Kurdi’s death shocks the
200,000 world
150,000 1 month after 1 month after 32 000 mentions 100,000 3 months before Alan Kurdi 26 000 mentions / week 7 000 mentions / week / week 50,000
0
Syrian Refugee conversation Alan Kurdi ACTIVITY HALF-LIFE – ALAN KURDI Moral Outrage has a much shorter half-life than general conversations
14,000 180 All Conversations Moral Outrage 160 12,000 Charlie Hebdo cartoon 140 10,000 120
8,000 3 Days 100
6,000 80
60 4,000 23 hrs 40 2,000 20
0 0 BEFORE & AFTER - ALAN KURDI SENTIMENT TOWARDS SYRIAN REFUGEES
Before Alan Kurdi: Change in sentiment: After Alan Kurdi: Negative: 26.1% Negative: -7.9% Negative: 18.2% Positive: 11.3% Positive: +14.0% Positive: 25.3% 70.0%
60.0%
50.0%
40.0%
30.0%
20.0%
10.0%
0.0%
Positive Negative ALAN KURDI SENTIMENT INTENSITY
Sentiment towards the death of Alan Kurdi Intense Sympathetic Outrage 13% 5%
Mild Outrage 82% OMRAN DAQNEESH BEFORE & AFTER ANALYSIS – OMRAN DAQNEESH
Bombing of aid trucks 1,600,000 in Allepo
1,400,000 1,200,000 Omran Daqneesh’s story goes 1,000,000 viral 800,000
600,000 1 month after 46 k mentions 400,000 1 month before Omran Daqneesh 450 mentions / week / week 200,000
0 7/4/2016 7/11/2016 7/18/2016 7/25/2016 8/1/2016 8/8/2016 8/15/2016 8/22/2016 8/29/2016 9/5/2016 9/12/2016 9/19/2016 9/26/2016 Syrian Refugee conversation Omran Daqneesh ACTIVITY HALF-LIFE – OMRAN DAQNEESH As with the story of Alan Kurdi Moral Outrage has a short half-life
20,000 700 All Conversations Moral Outrage 18,000 600 16,000
14,000 500
12,000 4 Days 400 10,000 300 8,000
6,000 25 hrs 200 4,000 100 2,000
0 0 OMRAN DAQNEESH SENTIMENT INTENSITY
Sentiment towards the assault on Omran Daqneesh Intense Sympathetic Outrage 1% 1%
Mild Outrage 98% PARIS ATTACKS BEFORE & AFTER - ALAN KURDI VOLUME OF CONVERSATION ON SOCIAL MEDIA Paris attacks 300,000
250,000
200,000 Alan Kurdi’s death shocks the world
150,000 1 month after 32 000 mentions 1 month after 100,000 3 months before Alan Kurdi / week 26 000 mentions 7 000 mentions / week / week 50,000
0
Syrian Refugee conversation Alan Kurdi BEFORE & AFTER – PARIS ATTACKS Press articles claiming that one of the attackers had a Syrian passport could’ve impacted the sentiment France (and Parisians in particular) had towards Syrian refugees in their countries. However, people were wary of accepting this until there was concrete evidence. In fact, France’s negative sentiment towards Syrian refugees actually decreased, with positive sentiment increasing by almost 10%! Negative Negative 9.5% 16.7% Positive Positive 30.8% 40.5%
BEFORE AFTER
Neutral Neutral 52.5% 50.0% MOST NEGATIVE ABOUT – PARIS ATTACKS 28.8% of Americans had negative sentiment towards Syrian refugees after the Paris attacks in November 2015. THE TRUMPETS Majority of this conversation was driven by Trump supporters. BRINGING IT ALL TOGETHER OVERALL VOLUME OF CONVERSATION
1,800,000 Bombing of aid trucks
1,600,000
1,400,000
1,200,000
1,000,000
800,000 Alan Kurdi 600,000 Paris attacks #IamSyrian 400,000 campaign Omran Daqneesh 200,000
0
2/9/2015 3/9/2015 4/6/2015 5/4/2015 6/1/2015 9/7/2015 2/8/2016 3/7/2016 4/4/2016 5/2/2016 8/8/2016 9/5/2016
1/12/2015 1/26/2015 2/23/2015 3/23/2015 4/20/2015 5/18/2015 6/15/2015 6/29/2015 7/13/2015 7/27/2015 8/10/2015 8/24/2015 9/21/2015 10/5/2015 11/2/2015 1/11/2016 1/25/2016 2/22/2016 3/21/2016 4/18/2016 5/16/2016 5/30/2016 6/13/2016 6/27/2016 7/11/2016 7/25/2016 8/22/2016 9/19/2016 10/3/2016
12/29/2014 10/19/2015 11/16/2015 11/30/2015 12/14/2015 12/28/2015 OVERALL CAMPAIGN ANALYSIS
Human-centred stories continue to appeal to emotions of citizens around the world and compel them to share these stories. People are drawn to narratives around the individual, particularly children.
Public outcry is most pronounced in 24hrs after an event. Knowing when this moment occurs can help better leverage this outcry for change.
Impactful photos or short videos appear to have the greatest impact and are shared prolifically. WOMEN IN REFUGEE CAMPS: MAY – JULY 2017 METHODOLOGY
Mentions from 1 Jan 2016 – 19 May 2017 Thematic Analysis: 1 Feb 2017 - 19 May 2017
811 k mentions about refugees
400 k unique authors (0 – 4 credibility rating) METHODOLOGY
Collected mentions in English
Mentions were collected from over 25 different countries OVERVIEW Contributors to conversation about refugee camps ranged from the general refugee population to those residing in or formerly resided in refugee camps. Male authors contributed to 34.7% of all conversation, whereas female authors contributed 16.9%.* Male Unknown Female
Conversation about Calais Similar to female conversation, but 17% refugee camp kitchens, with the addition of engagement with volunteers and the wellbeing of political discourse about the refugee 35% children. There was also crisis, with particular reference to conversation about initiatives Palestinians. by refugees within camps as well as about the conditions of Greek camps. 48% *Where gender could be determined. COUNTRY BREAKDOWN Conversation by country over data collection period UK: 71 922 Germany: 9 984
Canada: 17 052
USA: 148 530 Syria: 851
Egypt: 1 496 India: 19 881 Turkey: 7 940 RAPE AND VIOLENCE
In the first week of February conversation spiked due to rising concerns about the violence and rape perpetrated against women. The conversation focused primarily on cases in Germany and Britain’s Dunkirk refugee camp – which would have a fire in April resulting in the displacement of refugees.
16000 1600
14000 1400
12000 1200
10000 1000
8000 800
6000 600
4000 400
2000 200
0 0 1/30/2017 2/28/2017 3/31/2017 4/30/2017 Conversation (excl. rape) Rape SYRIA CONVERSATION
Conversation within Syria
Aleppo 2
Homs 2
Conversation about Syria
The Magic in the • Understanding public sentiment Machines towards a company, organisation, person or event is useful information.
• Understanding the detail of the topics that are driving that sentiment is social intelligence that can manage risk and guide strategic decision-making.
COLIN HABBERTON @RELATOMICS +27 71 401 2434