Project Objectives

ABSTRACT

Emotions are central to politics and advertising. The American society is going through a divided period after Trump’s election. Despite political differences, people are very much concerned about his impact on people’s lives. Since the 2016 presidential election, Clinton and

Trump have been using emotionally appealing messages to rally support via every channel. To understand what people feel and how they feel, this paper will analyze emotional reactions from comments on Trump’s inauguration speech, the executive order on immigration policy, and the joint address to Congress. The goal is to find out how people feel compared to their expectations within the first 100 days of the Trump administration. Big data analysis and emotion analysis will be conducted on the most recent 3,000 comments for each video. Each comment will be labeled from the following five categories: anger, disgust, fear, joy, and sadness through IBM bluemix platform. Then the result will be shown in data visualization format (word cloud, network analysis, pie charts, etc) for conclusion. The research will also cross reference to policies related to the three videos and executive orders Trump signed from January 20, 2017 to

April 29. The final report presents a holistic picture from actions to words, from emotions to policy making, and from provocative issues to a forecast. INTRODUCTION

Justification

According to globenewswire [1], worldwide emotion analytics market will grow to

82.9% during the forecast period 2016-2022 to aggregate $1,711.0 million by 2022. Although emotion analytics is a niche market, many early stage startups are entering the market and focusing on analyzing customer emotions to offer better user experiences. This research will create news value in journalism, political critiques, and NLP research. However, rare academic projects or news outlets have utilized NLP (natural language processing) algorithms to conduct emotion and sentiment analysis on a trendy political event. The limitation of this project depends on the selected platform and tools used. For instance, YouTube comments may contain spams generated by chatbots programs. This creates noise in the data and interferes with the results.

Compared to traditional research methods such as questionnaires, focus groups, and surveys, the big data research method is more efficient, understandable, and usable to look at and produce logic insights. I hope to create a more quantitative research project within the Emerson community and it will be a stepping-stone to my interest in the big data, emotion artificial intelligence, and information visualization field.

Project objectives

April 29th is Trump’s 100th days in the office. The goal of this project is to find out how people feel compared to what happened and to what degree did people feel mostly challenged on each issue (on either positive or negative scale). Another layer to the research is to look at what types of words, wording, and structures are trending. Hypotheses

The hypothesis will be based on the most valid information I gather. More detailed labels on each dataset will provide the best backup for each hypothesis. For example, within 3,000 comments of the inauguration video, people with a higher education background are labelled as more disappointed than those who are less educated which indicates the correlation between education and sentiment lean. If the labelled emotion is too small to be considered statistically significant, then it will reject the hypotheses (null). There are five assumptions as followed:

1. The extent to which people felt aggressively on the immigration issues is

contingent to their citizenship and languages they’re posting.

2. The degree of negative sentiments should correlate with the time when Trump

signed the executive orders.

3. The emotion fluctuations should show peaks surrounding the most popular (most

frequent and most cited) keywords from Trump’s speech.

4. Within the 3,000 comments, opinion clusters should form varied by party

affiliation and attitudes on each side.

5. The text analysis and emotion charts should show a common prevailing negative

response.

Research questions

Research questions: After examining few related researches on Youtube comments [10], I raised following questions:

● What are the typical emotional reactions on each of Trump’s YouTube videos?

● Which issues do people react to most? How did non-english speaking people react

compared to U.S citizens who posted in English. ● What are the key words and factors that trigger discussions on YouTube videos?

● How long after the video came out did it trigger the most discussion (either negative or

positive)?

● What’s the comparison between the sentiment in the comment and the sentiment from

Trump’s own speech text?

● Which emotion are people showing the most?

LITERATURE REVIEW

Previous work has shown that consumer brands and advertising agencies are eagerly bringing data to real ROI. Possible, an advertising agency, [2] presented research on measuring emotion in a volatile election, showing that “people are much more broadly aligned in negative sentiment toward Trump than Clinton.” Emotion artificial intelligence companies such as

Tvision, Affectiva, and Beyond Verbal have done similar research on the emotion of the audience in front of TV, mobile screens, and YouTube respectively. McDuff, Kodra, and Picard, the co-founder of Affectiva [3] found out they were able to build a model to measure voter’s candidate preference based on affective responses to presidential live debates (Obama vs.

Romney). The results showed different responses can be detected from viewers with different political preferences and that similar expressions at significant moments appear to have very different meanings. Heubl [4] provides a facial recognition API technology in analyzing Trump and Clinton’s emotions during debates and found that Clinton’s smile is a careful attempt to avoid mistakes while Trump tried everything to pry out an honest emotional response. Brader [5] found that political campaigning achieves its goal in part by appealing to the emotions of the viewers and that different emotional states led to different self-report responses. There is plenty of research on text analysis for Twitter as well. Dahlke [6] wrote a python script to collect tweets and retweets that contain the words Hillary and Trump directly from twitter.com while Robinson

[7] analyzed Trump’s Twitter post using R and saw that the Android tweets are angrier and more negative, while the iPhone tweets tend to be benign announcements and pictures. Meanwhile,

Semantic Visions [8] built a sentiment analysis chart for data visualization to show that sentiments towards Trump’s speech were more negative than Clinton’s. Although the algorithm each company had was impressive and able to run a large amount of data, the small margin of error defined the result of the election. Cognovi Labs [9] offered valuable insights on how Trump won socially before he won the campaign.

On the other hand, politicians, parties, and campaign managers are only at the surface level to utilize big data analytics well. Major goals include: one, to accurately target eligible voters and use that information to optimize their spending on media–those all-important TV ads, in particular. Two, to attract bigger sums of money for campaign advertising. The company Deep

Root was born out of efforts by the Republican Party to compete more effectively with

Democrats on big data analytics. The company is essentially the big data arm of the GOP, providing data and analytic services to Republicans running for office–from the president on down to state legislative races. Both Deep Root and TargetSmart use software from Alteryx to help them ingest, cleanse, blend, and analyze massive amounts of data from a variety of sources.

On the contrary, developers and researchers are far more advanced in the application. Netlytic, a cloud-based text and social networks analyzer developed by Social Media Lab at Ryerson

University in Toronto, was used in my research. It offers a way to capture data from social media sites without users having any programming and API knowledge. It can discover popular topics, find, explore emerging themes of discussions, build and visualize livestream social networks using social network analysis. Scholar Anisa Awad discovered the communication network between Twitter users by analyzing 63,430 tweets during live Oscar ceremony in 2014. She was able to identify top five clusters surrounding @TheAcademy, with the ellen show being the largest cluster. Other machine learning tools and libraries such as Spark and IBM Watson also serve as a strong support on accuracy level. If tools like Netlyic are properly used in political campaigns for messaging and engagement, Hillary Clinton may not lose on the election day. METHODOLOGY

Data collection

Trump’s inauguration speech was given on January 20 via ABC news, attracting

3,791,899 views. The video on “New Immigration Policy And Border Wall Details” streamed live on January 25 via FOX after Trump signed two executive orders on border security and immigration policy. The third video is the “Joint Address To Congress” on February 28 on NBC news. Total 9,000 comments for three selected videos were collected on April 9th through web scraper/YouTube API and stored as separate JSON files. The source of the video came from major television networks’ YouTube channel. I used an online web scraper called “ytcomments” developed by Philip Klostermann. Variables include userID, comments, timestamp, replies and likes. The raw excel files are sorted by date and only the most recent 3,000 comments were selected. After deleting the duplicates and empty cells, the excel data was converted to JSON file and parsed through IBM’s alchemy Language to be labeled with categorical emotions. To process labeling function, I consulted my Ph.D. friends at MIT for the python language writing.

The emotion score for each category ranges from 0 to 1. Sum and average scores were calculated and ready to visualize. Meanwhile, Semantria, a tool for sentiment score analysis, ran the three sets of data. Semantria returned files with detected entities, themes, languages, and sentiment score (calculated from emotion scores) for each comment. The sentiment score then went through JMP (a statistical tool by SAS) to produce a logistic fit curve to show distribution. The limitation is that Semantria can only read 100 queries with 1500 characters on each query text so longer comments were filtered. The number of queries for the inauguration, immigration, and congress speech was 2849, 2862, and 2899 respectively. Netlytic then drew word-over-time graphs and popular keyword cloud. It migrated data into network analysis which contains name- network and chain network diagrams. After seeing the peaks on each visualization, I looked back on that day to see what exactly happened. To be more specific, which executive order did Trump signed and what stories were related. Name network diagrams give me a view on opinion cluster

- people who shared similar views on each topic are grouped into one cluster. I am able to manually find these comments and draw conclusions from featured comments.

Metrics and measurements

Text analysis

Wikipedia[11] defines text analysis as “a set of linguistic, statistical, and machine learning techniques that model and structure the information content of textual sources for business intelligence, exploratory data analysis, research, or investigation”. The higher-level goals in this method are to investigate language usage, entity recognition/extraction and visualization. Programming language skills are preferred but not required because tools are the means not the end. Toolsets for text-mining in this project include JMP (word frequencies, word cloud), Bitext (keyword extraction), Netlytic (network, chain analysis) and IBM Watson analytics, Gephi, and Tableau for visualization.

Network analysis

In addition to text analysis, I also conducted network analysis via Netlytic which provides the most basic interactive network analysis involving building networks from members

(‘network actors’) connected together based on some common form of interaction (‘ties’). When building networks from interaction data, however, there are a lot of different parameters and threshold choices to choose from. For example, one of the choices that is likely to influence network formation is how to discover ties between individuals. Netlytic approaches this task by building two types of social networks: (1) Name network and (2) Chain (reply-to) network. In the networks, each node represents a name, a person, or a comment. The ties in between them represent the relationships between two entities. The shorter the closer. Nodes form clusters and clusters indicate where opinions lean and whose comments attract most interactions.

Semantria for sentiment analysis

Semantria categorizes results into sentiment polarity and sentiment scores. Polarities can be "negative", "neutral", or "positive". Scores have two types - Component sentiment and document sentiment depends on target entities. Components are themes, topics, and entities. The score range is from -10 to 10. The document sentiment has a range spread from -2 to 2, where -2 is really negative, -1 is negative, 1 is positive, and 2 is really positive.

● Component sentiment: Negative < -0.45, Positive > 0.5

● Document sentiment: Negative < -0.05, Positive > 0.22 (semantria manual, 21)

Common limitations

Several commercial and academic tools, such as those from IBM analytics, SAS, Oracle,

SenticNet, and Luminoso, track public viewpoints on a large scale by offering graphical summarizations of trends and opinions. Yet most COTS tools are limited to a polarity evaluation or a mood classification, according to Erik Gambria [13], a scholar from Nanyang Technological

University. Human emotions are generally categorized to a limited set of emotions in each research platform and such methods rely mainly on parts of text in which emotional states are explicitly expressed and, hence, they cannot capture opinions and sentiments that are subtle and implicit in YouTube comments.

YouTube: chatbots refer to artificial conversational entities that conduct conversations and post comments by a computer program. A classic example of the malicious use of chatbots was Yahoo messenger’s inability to deal with the spam unleashed by its bots. Arun Uday pointed out in a Techcrunch article, “In fact, so bad was the problem that it forced Yahoo to contrive the now familiar captcha code to prevent bots from automatically entering chat rooms” (Uday, 35).

While chatbots inspired academic researchers to develop techniques for distinguishing bots from humans, there’s no record showing any programming languages can successfully detect them with 100% accuracy.

IBM: The latest version of tone analyzer was released in July 2016. Tones detected within the General Purpose Endpoint include joy, fear, sadness, anger, disgust, analytical, confident, tentative, openness, conscientiousness, extraversion, agreeableness, and emotional range.

Though humans emotions are far more than these categories, I’m using the basic five emotions to start the journey. IBM bluemix free accounts can process up to 1,000 queries everyday.

Excel to Json: When YouTube comments contain emojis or foreign languages, JSON files can store them but excel file won’t be able to convert it into Unicode. The issue increased the number of comments filtered out by Semantria and Netlytic. In this project, I manually read excel files and delete the invalid characters and emojis. A debug python program may help to solve the issue faster but it requires extensive programming knowledge for researchers.

Semantria: As mentioned earlier, Semantria has the following input limits:

● Configurations/Profiles - 10 configurations/profiles allowed with up to 50 characters per

configuration/profile

● Blacklist - 100 items of up to 50 characters allowed

● Queries - 100 queries allowed with up to 50 characters title and up to 1500 characters

query text. ● Categories - 100 categories allowed with up to 50 characters titles and up to 10 samples

per category (sample can be up to 50 characters as well)

● Entities - 1000 entities allowed with up to 50 characters per title and entity type.

● Sentiment-bearing Phrases - 1000 phrases allowed with up to 50 characters per phrase

● Maximum size of document ID - 36 characters

● Allowed document size - 2048 characters per document

● Incoming batch size - 100 documents per batch

● Outgoing batch size (including web hooks) - 100 documents per batch

● Allowed collection size - 1000 documents per collection

Netlytic: Though Netlytic builds powerful networks among members when the number of name nodes exceeds than 1,500, it slows down the rendering speed (also depends on computers’

Graphic Processing Unit) dramatically, making it harder to analyze larger datasets. In addition, each free trial account can only process up to five datasets.

Professor Gambria said, “Next-generation-sentiment-mining systems need broader and deeper common and commonsense knowledge bases, together with more brain-inspired and psychologically motivated reasoning methods, to better understand natural language opinions”

(Gambria, 106). NLP machine learning methods should hence evolve with how the next generation speaks, comments, and interacts on social media.

FINDINGS

The inauguration video Figure 1. Word cloud from inauguration dataset. Figure 2. A pie chart based on average sentiment score data. 3,000 records were analyzed.

The word cloud shows the most-mentioned keywords in comments from the inauguration speech video. Interestingly, the most frequent words are “Trump”, “bang”, “people”. The color palette of blue to yellow shows the frequency. The pie chart shows the “disgust” is the prevailing feel among people. In addition, Netlytic also provides a stacked graph for textual analysis. The interactive chart below shows the top 37 topics from January 20 to January. The number of topics displayed range from

10 to 100. The visualization map below demonstrates those trending keywords in terms of their popularity over the period of time. For the purpose of making the visualization map as useful and insightful as possible, certain non-English words and irrelevant such as “certo”, “pouco”,

“shinzo”, and “imundo” were removed. Examining the timeline and the speech, main themes are

“Trump copied sentence structures from Barry Benson - the main protagonist from Bee Movie”,

“what exactly is to rebuild the American Dream” when Trump mentioned the campaign slogan

“Make America Great Again”. The keyword “colony” shows the discussion around Trump questioning Obama’s birthplace and thank him and Michelle Obama for their work. The attitude shows people think Trump is a hypocrite. When Trump mentioned “The forgotten men and women”, comments show a “clichés phrase” for Republican presidents who like referring to the “silent majority,” coined by Richard Nixon and won over by Reagan.

Figure 4. Figure 5.

Two charts above show the logistic analysis by JMP. With a chi square of 638.1 and a probability of less than .0001, we can conclude that there is a relationship between the sentiments degree and the time. P value suggest the true confidence is larger than 99.99% which proves the hypothesis 2 is highly significant. Blue line indicates the attitude changes and black dots mark each comment and concentration. The peak around 31, 32 and 60 days after the video release date guided me to check which document Trump signed. On February 28 he signed

Restoring the Rule of Law, Federalism, and Economic Growth by Reviewing the "Waters of the

United States" Rule. Featured comments include “You want a wall but not clean water in flint

Michigan” (Anthony Feliciano), “I would say just like Nazi Germany, but they were actually better than us since there was no campaign to destroy Germany's clean water and clean air and there was no nurturing of rabid hatred against German women” (NaomiOne1).

The new immigration policy and border wall video

Figure 6. Figure 7.

Trump updated the details on building Mexican wall to enforce border security on the first travel ban on January 25, emphasising “a nation without border is not a nation” in the speech. Followed by the second travel ban on the 27th explicitly bans the entry from Iraq, Syria, Iran, Libya,

Somalia, Sudan and Yemen. Though both executive orders were rejected by federal judges, they created huge chaos in the airports, discussions online, and mass protests on the streets. Figure 7 shows 25.8% people feel sad about what he said and done. The top four peaks in figure 8 are

Figure 8. Words over time during the week of January 26 “wall” (green), “Trump” (pink), “people” (cyan), and “country” (purple). When I increased the

Figure 9. Words over time during the same time with 100 unique topics themes to 100. The unique comments mentioning Trump increased 10%. Semantria enabled me to further look at what’s breakdown of citizenships and languages used. Out of 2,862 valid comments, 51% are English, 9 non-English languages were detected, containing most European languages. Within the 49% non-English comments, 18% are unknown languages due to

Semantria NLP feature cannot detect Arabic languages. The document sentiment score is 0.002 and component score is - 0.017. Both scores are within the “neutral” range. Figure 10 clearly explains why the sentiment is “neutral” instead of “negative”. The positive scores and negative scores split into two polarities as time went by. Furthermore, logistic fit indicate the number of comments increased from the 5th, 14th, 21th day to reach the first peak on the 31the day, and reached the highest peak on two month after the first travel ban. Since judges in New York and

Massachusetts issued restraining orders in late January, travel ban had setbacks from Sally Yates, James Robart, Derrick Watson, Theodore Chuang, etc. With a chi square of 1616.43 and a probability of less than .0001, we can conclude that there is a highly significant relationship between attitudes and immigration issues over time.

Figure 10.

Then Netlytic network analysis showed me where the opinion clusters were.

Figure 11. Figure 12. Figure 10.

DISCUSSION CONCLUSION

REFERENCES

1. Wood, Laura. "Worldwide $1.71 Billion Emotion Analytics Market 2016-2022: Drivers,

Opportunities, Trends, and Forecasts - Key Players are Microsoft, IBM, Retinad VR,

Neuromore, Imotions, Kairos, Affectiva & Eyris." Nasdaq globenewswire.13 Jan. 2017.

Web. 16 Feb. 2017

2. Cobb, Michael. "Measuring Emotion in a Volatile Election." Possible. 07 Nov. 2016.

Web. 30 Jan. 2017.

3. McDuff, Daniel, Rana El Kaliouby, Evan Kodra, and Rosalind Picard. "Measuring

Voter’s Candidate Preference Based on Affective Responses to Election Debates."

Affective Computing. 02 Sept. 2013. Web. 30 Jan. 2017.

4. Heubl, Ben. "How to apply face recognition API technology to data journalism with R

and python." Data dico. 20 Oct. 2016. Web. 30 Jan. 2017.

5. T. Brader, “Striking a responsive chord: How political ads motivate and persuade voters

by appealing to emotions,” American Journal of Political Science, vol. 49, no. 2, pp.

388–405, 2005. 30 Jan. 2017.

6. Dahlke, Dan. "Election 2016: Analyzing Real-Time Twitter Sentiment with MemSQL

Pipelines." MENSQL . 18 Oct. 2016. Web. 31 Jan. 2017.

7. Robinson, David. "Text analysis of Trump's tweets confirms he writes only the (angrier)

Android half." Variance explained. 09 Aug. 2016. Web. 29 Jan. 2017.

8. "US PRESIDENTIAL ELECTION 2016 Sentiment Analysis Charts." Semantic visions.

2016. Web. 30 Jan. 2017.

9. Donovan, Jay. "Cognovi Labs is watching the U.S. Presidential Election with its

predictive, social sentiment tool.” Tech Crunch. 08 Nov. 2016. Web. 29 Jan. 2017. 10. Thelwall, Mike, Pardeep Sud, and Farida Vis. "Commenting on YouTube videos: From

Guatemalan rock to el big bang." Journal of the American Society for Information

Science and Technology 63.3 (2012): 616-629.

11. "Text mining." Wikipedia. Ed. Wikipedia. 19 Jan. 2017. Web. 16 Feb. 2017.

12. "Introduction to Text Analysis: Analysis Methods and Tools." Duke university library.

Ed. Duke. 19 Jan. 2017. Web. 17 Feb. 2017.

13. Cambria, Erik. "Affective computing and sentiment analysis." IEEE. Mar. 2016. Web. 17

Feb. 2017. https://www.datanami.com/2016/05/10/winning-politics-now-tied-big-data/ https://netlytic.org/home/?p=171 https://techcrunch.com/2016/07/16/bursting-the-chatbot-bubble/ https://en.wikipedia.org/wiki/Chatbot#Malicious_use https://www.ibm.com/blogs/bluemix/2016/10/watson-has-more-accurate-emotion-detection/

Gruzd, A. (2016). Netlytic: Software for Automated Text and Social Network Analysis. Available at http://Netlytic.org http://support.semantria.com/customer/portal/articles/834168-about-semantria-s-sentiment- analysis SPECIAL THANKS

Jinni Luo Northeastern University, Master’s Degree, Information Design and Data

Visualization

Fangchang Ma Massachusetts Institute of Technology, Doctor of Philosophy (Ph.D),

Autonomous Systems

Tian Xia Northeastern University, Master’s Degree, Computer Science

Emma Sheng Boston University, Master’s Degree, Actuarial Science