Sentiment Patterns in Videos from Left- and Right-Wing Youtube News Channels

Uphill from here: Sentiment patterns in videos from left- and right-wing YouTube news channels Felix Soldner1, Justin Chun-ting Ho2, Mykola Makhortykh3, Isabelle van der Vegt1, Maximilian Mozes1,4 and Bennett Kleinberg1,3 1University College London 2University of Edinburgh 3University of Amsterdam 4Technical University of Munich 1{felix.soldner, isabelle.vandervegt, bennett.kleinberg}@ucl.ac.uk [email protected], [email protected], [email protected]

Abstract Netherlands) (Newman et al., 2018). The reasons for the growing online consumption of news are News consumption exhibits an increasing shift many: digital outlets provide higher interactivity towards online sources, which bring platforms such as YouTube more into focus. Thus, the and greater freedom of choice for news readers distribution of politically loaded news is eas- (Van Aelst et al., 2017), and the possibility to ac- ier, receives more attention, but also raises the cess them through mobile devices enables more concern of forming isolated ideological com- flexibility and convenience in consuming news munities. Understanding how such news is content (Schrøder, 2015). communicated and received is becoming in- Despite the multiple benefits associated with creasingly important. To expand our understanding in this domain, we apply a linguistic "the digital turn" in news consumption, it also temporal trajectory analysis to analyze senti- raises concerns regarding possible changes in the ment patterns in English-language videos from societal role of news media (Coddington, 2015). news channels on YouTube. We examine tran- It has been established that there is a relationship scripts from videos distributed through eight between news consumption and the formation of channels with pro-left and pro-right political public agendas, including political attitudes of the leanings. Using unsupervised clustering, we general population (McCombs, 2014). However, identify seven different sentiment patterns in the possible impact of digitalization on these re- the transcripts. We found that the use of two sentiment patterns differed significantly de- lationships remains a subject of academic debate pending on political leaning. Furthermore, we (Helberger, 2015; Eskens et al., 2017; Van Aelst used predictive models to examine how differ- et al., 2017). An increasing number of stud- ent sentiment patterns relate to video popular- ies examine the possible connection between on- ity and if they differ depending on the chan- line news consumption and the formation of iso- nel’s political leaning. No clear relations be- lated ideological communities, which can nur- tween sentiment patterns and popularity were ture biases and limit citizens’ societal participa- found. However, results indicate, that videos from pro-right news channels are more pop- tion (Gentzkow and Shapiro, 2011; Flaxman et al., ular and that a negative sentiment further in- 2016; Sunstein, 2017). Such formations can lead creases that popularity, when sentiments are to increased political or personal radicalization, averaged for each video. which is a concern for societal cohesion. Keywords: linguistic temporal trajectory Polarization concerns related to audience seg- analysis, online news, left-wing, right-wing, mentation are amplified by the use of affective lan- sentiment analysis, YouTube guage for producing online news stories (Soroka et al., 2015). Despite the widespread belief that 1 Introduction news stories tend to use balanced and unbiased Today, news is increasingly consumed through on- language, recent studies suggest that this is not line platforms. Approximately nine out of ten necessarily true, in particular in the case of news adults (93%) in the US acknowledge reading news stories crafted for online environments such as online (Center, 2018); similar, if slightly lower news websites (Young and Soroka, 2012). A num- rates, are observed for many European coun- ber of studies suggest that the use of affective tries (e.g. 65% of Germany and 79% in the language can increase audience engagement with

84 Proceedings of the Third Workshop on Natural Language Processing and Computational Social Science, pages 84–93 Minneapolis, Minnesota, June 6, 2019. c 2019 Association for Computational Linguistics news content (e.g. stories with a negative tone The growing adoption of sentiment analysis as seem to be more popular (Trussler and Soroka, a research tool is accompanied by methodologi- 2014; Soroka et al., 2015)). At the same time, the cal improvements. Originally employed for clas- relationship between a narrative’s popularity and sifying user reviews coming from the commercial the specific patterns of sentiment remains under- domain (Pang and Lee, 2008), early approaches investigated, as well as the ways this relation- to sentiment analysis were focused on producing a ship varies among audiences with different politi- single sentiment score for the specific document or cal leanings. a text section using binary assessments of polarity. In this paper, we contribute to the debate on Such approaches, however, resulted in rather sim- the use of emotionally loaded language in on- plified evaluations, which reduced sentiment com- line news stories as well as its variations between plexity to binary constructs under which the whole outlets with different political leanings. Specif- document could be either positive or negative or ically, we examine the sentiment trajectories of (in some cases) neutral. This can lead to incorrect YouTube news channels related to major English- conflations of sentiment variations within a text. language media outlets. Our interest in YouTube The response to these limitations included the is attributed to the platform’s significant impact advancement towards more fine-grained assess- on information consumption as users increasingly ments of sentiment, including the identification of consume news in the video format (al Nashmi a more complex spectrum of emotions (Cambria et al., 2017). Together with Facebook, YouTube et al., 2015), but also the transition towards the dy- is the main platform used by consumers to watch namic assessment of sentiment shifts throughout news outside of news organizations’ own websites texts (Jockers, 2015; Tanveer et al., 2018; Klein- (Newman et al., 2018). News content produced by berg et al., 2018). news organizations for YouTube usually follows Until now, the sentiment of news stories re- traditional broadcast standards (Peer and Ksiazek, mains a rather under-investigated subject. Kaya 2011) and formats (al Nashmi et al., 2017) and et al.(2012) note that unlike user reviews of prod- often serves as an extension of their distribution ucts (e.g. movies), news stories are considered model. At the same, YouTube also enables ex- to be written in a neutral way. A different study perimenting with new formats and engaging au- found that news stories about the economy and diences in more interactive or provocative ways the environment were overall more positive than (al Nashmi et al., 2017). about crime or international topics, which were Using YouTube as a case platform, we test for overall more negative (Young and Soroka, 2012). differences in the dynamic use of sentiment in Despite the growing number of studies on news news coverage between channels leaning to the sentiment, only a few of them so far approach it political left (e.g. CNN) and right (e.g. Fox by considering the dynamic shifts of sentiment. News). Additionally, we examine if sentiment pat- A study, examining sentiments of different top- terns have predictive value for news videos’ popu- ics (e.g. earthquakes) included time stamps over larity on YouTube. a three months period, to model the sentimental change of news stories (Fukuhara et al., 2007). 2 Related Work In that way, they were able to show a sudden in- Since the mid 2000s, sentiment analysis remains crease of negative emotions in news corpora, when one of the fastest growing fields of machine learn- an earthquake occurred. These negative emotions ing and computational linguistics (Mäntylä et al., slowly decreases over time. While this approach 2018). Defined by Liu(2010) as a collection of models sentiment change over time, it cannot ac- methods for detecting and extracting subjective count for shifts within single stories. information (e.g. opinions, attitudes and emo- Among multiple areas of research on news story tions) from language, sentiment analysis is in- sentiment, the issue of variation in the use of affec- creasingly adopted for multiple academic areas, tive language by news outlets with different politi- varying from from media studies (Sivek, 2018) cal leanings (e.g. left- or right-wing) is both under- to literature (Gao et al., 2016) to political sci- investigated and urgent. The language of politics- ence (Cambria, 2016) and conflict studies (Welch, related texts does not only reflect, but can also 2018). influence the sentiments of the audience (Brader,

85 2005). A number of studies point to the dis- 3.1 News channels selection tinct features of online political communication The data consist of all English-language chan- depending on the actors’ political leaning (e.g. nels from the top 250 news channels on (Engesser et al., 2017; Bracciale and Martella, YouTube, which were ranked by SOCIAL- 2017; Hameleers et al., 2017; Schoonvelde et al., BLADE3 (www.socialblade.com, retrieved 2019a)). Engesser et al.(2017) identify that social November 2018). From that pool, 18 news chan- media usage of right-wing political actors is char- nels were selected, which were identified as the acterized by the use of a few key features, such as ones holding political bias (i.e. either left- or emphasizing the sovereignty of the people, advo- right-wing) by Media Bias/Fact Check4. The web- cating for the people, attacking the elite, ostraciz- site uses a rating method to identify biases among ing others, and invoking the concept of a "heart- information sources and has been used by pre- land". Similarly, Bracciale and Martella(2017) vious studies dealing with media bias (Bentley show that communication styles of populist actors et al., 2019; Bovet and Makse, 2019; Mehta and tend to involve highly emotional language and that Guzmán, 2018). they are particularly keen on referring to negative emotions (e.g. fear) to mobilize their supporters. 3.2 Obtaining video transcripts The studies mentioned above, however, focus Video transcripts were scraped with the help of on political statements produced and distributed "www.downsub.com", which retrieves the tran- through different media; yet, little is known how scripts of specific YouTube video URLs, and different political sentiments materialize in online the "beautifulsoup" python package (Richardson, news stories, in particular in the YouTube video 2019) (for more details see Kleinberg et al. format, which is the major focus of our study. A (2018)). Downloaded transcripts were manu- number of studies discuss the impact of news out- ally or automatically generated. Videos with- let ideological leanings on the way specific sub- out transcripts were not included in further anal- jects are covered (de Vreese, 2005). Most of these yses. Retrieved transcripts were cleaned by re- works, however, tend to focus on traditional for- moving XML tags and merged into one string, mats of news stories (i.e. text) and use qualitative without punctuation for each video. Selected tran- approaches to examine coverage of a specific sub- scripts for further processing adhered to the fol- ject, such as climate change (Dotson et al., 2012; lowing criteria: at least 100 words; at least 50% Feldman et al., 2012) or protest campaigns (Ha of words are matched English words; at least 90% and Shin, 2016; Shahin et al., 2016). In our paper, of words are ASCII-encoded; and from channels we propose to look at the intra-textual dynamics with more than 2000 valid transcripts. Subse- of the overall sentiment of content produced by quently, for 4 left and 4 right channels (randomly the outlet in question and employ a quantitative selected) 2000 transcripts were selected, 7 non- approach to trace if there are differences between English transcripts from Business Insider and Rus- right- and left-wing news outlets. sian today were then excluded, resulting in a bal- 3 Method anced dataset of 15993 transcripts (table1). The data used in this study are publicly available1 3.3 Popularity rating and the current work is the joint product of a work- Since we are interested in the popularity rating of shop on linguistic temporal trajectory analysis at each video and its association to sentiment style, the European Symposium Series on Societal Chal- we created an adjusted popularity rating. We cal- lenges in Computational Social Science in 2018. culated a popularity index defined as the num- This includes the pre-processing and feature ex- ber of upvotes divided by the total views for each traction2 of the data, which is needed for the anal- video, which would be a score between 0 (no up- yses we performed in the current study. The data votes) and 1 (upvotes is equal to views). The ad- has not been used in other research and was specif- justment allows us to compare the popularity be- ically collected to devise the current work. 3SOCIALBALDE is an online platform, using data from 1Data: https://github.com/ben-aaron188/ different online platforms, such as Youtube, to create statis- ltta_workshop tics and rankings of these platforms and their content. 2Code for feature extraction: https://github. 4For more details, see the project’s website, https:// com/ben-aaron188/naive_context_sentiment mediabiasfactcheck.com.

86 tween videos, which have different upload dates, Channel Pol N. of videos Avg. wc AI Jazeera English Left 2000 1000.49 because videos available over a prolonged period Business Insider Left 1997 501.56 of time have the advantage of accumulating more Fox news channel Right 2000 777.85 upvotes. MSNBC clean forward Left 2000 1499.25 Russia today5 Right 1996 1422.71 The young turks Left 2000 1504.59 3.4 Feature extraction The daily wire Right 2000 4845.80 The organizers of the workshop provided us Rebel Media Right 2000 1250.73 with the necessary features for further analyses, Table 1: YouTube news channels distribution; Pol = who were inspired by Jockers(2015) and Gao Political leaning; N. = number; wc = word count et al.(2016); (for more details see Kleinberg et al.(2018)). Features were generated from the transcripts, which capture the sentiment change leanings (table1). throughout the transcript. The applied method is 4.2 Clustering based on the approach of the R package "sentimentr" (Rinker, 2019a), which generates senti- In order to examine sentiment patterns within the ments on a sentence level, but the current approach YouTube videos and their predictive value on pop- extends it to continuous text without punctuation ularity, we examined whether overarching senti- as is the case with video transcripts. The "naive ment patterns are present. We used an unsuper- context" sentiment extractor (Kleinberg et al., vised k-means clustering method and determined 2018) accounts for valence shifters, which influ- the number of k through the within cluster sum of ence the meaning of the sentiment. Negators (e.g., squares inflexion method (Thorndike, 1953) for 1 not, doesn’t), [de-]amplifiers (e.g., really, hardly), to 30 clusters (figure1). Following this, we used and adversative conjunctions (e.g., but, however) a k-means method with k = 7, which resulted in were included. This is important when generating seven clusters, each representing a sentiment be- the sentiments for sentences like "I had a really havior pattern found in the transcripts. All the pat- good day" (amplifying "good" through "really") terns are displayed in figure1, showing the aver- or "My day was not bad" (changing "bad" from a age sentiment trajectory for the adjusted timescale negative to a positive sentiment through "not"). In from 0 to 100, and sentiments from −1 (most neg- order to extract sentiment values, each word is as- ative) to +1 (most positive). The patterns corre- signed a sentiment value based on the "Jockers and spond well with the patterns found in related work Rinker Polarity Lookup Table" from the lexicon R (Kleinberg et al., 2018). Thus, we assigned the package (Rinker, 2019b). For the extractions of corresponding taxonomy names to our patterns. the features a window approach is taken, by which The dotted blue line indicates the average behavior the sentiment of the core word and its surround- pattern for the cluster and the red lines indicate the ing words of +/ − 3 are considered. This cluster standard deviation of +/ − 1. Table2 shows the of seven words is assigned an adjusted sentiment taxonomies and descriptive statistics of the senti- value by calculating the product of all the sentiment clusters. ment values in the cluster. The features are represented in vectors, containing zeros and weighted 4.3 Sentiment styles and political stance sentiment values. Theses values are standardized We also examined whether there is a relationship to a narrative time from 0 to 100, with a discrete between political stance and sentiment clusters. A cosine transformation from the "syuzhet" R pack- significant association was found with a 2 (po- age (Jockers, 2015) and scaled from −1 to +1 (i.e. litical stance) by seven (cluster) Chi-square test lowest sentiment (negative) per transcript to high- (χ2(14) = 25.31, p < 0.001). Table3 shows est sentiment (positive) per transcript). that the cluster "Downhill from here" was significantly p < 0.01 more used by politically right 4 Results leaning news channels. The reversed effect was observed for the cluster "Uphill from here", which 4.1 Data The total data set consists of 15993 video tran- 5Russia today (RT) can be considered as having a right- leaning political political stance at its core. However, this scripts, which are distributed between eight news can have some exceptions, such as supporting the yellow vest channels, and are equally divided into political movement in France.

87 88

Figure 1: Cluster plot and sentiment behavior patterns for each cluster. Cluster Description N. of videos % of videos Avg. vc Avg. up v. Rags to riches Negative curve turns into positive curve 2675 16.73 827.00 15.53 Riches to rags Positive curve turns into negative curve 2587 16.18 1002.47 17.11 Downhill from here Short positive turns into consistent negative 2177 13.61 919.94 16.82 End on a high note Short negative turns into consistent positive 2194 13.72 928.78 17.03 Uphill from here Consistent negative turns into short positive 2085 13.04 823.17 14.37 End on a low note Consistent positive turns into short negative 1547 9.67 846.80 16.81 Mood swing Small positive start into negative-positive- 2728 17.06 910.83 16.57 negative curves with small positive ending Total All 15993 100 897.71 16.32

Table 2: Sentiment styles taxonomy (adopted from Kleinberg et al.(2018)) and descriptive statistics; Average (Avg.) scores are adjusted by the number of days the videos were uploaded; N = Number; vc = view count; v = votes.

Political leaning a low note" (β = .02, se = 1.87, p = .99). Nei- Cluster Left Right ther model explains a sufficient proportion of the Rags to riches -0.96 0.96 variance to be considered informative, R2 = 0.00 Riches to rags 1.09 -1.09 and R2 = 0.03, for the first and second model, Downhill from here -3.25* 3.25* respectively. End on a high note 1.28 -1.28 We also split the transcripts in three equal sized Uphill from here 2.74* -2.74* components (beginning, middle, and end) and cal- End on a low note -2.44 2.44 culated the average sentiment rating for each part Mood swing 1.13 -1.13 and used a OLS regression model to test for an ef- Table 3: Chi-Square residuals; * = statistically signifi- fect of the components on the adjusted popularity 2 cant (α = 0.01). (F (10, 15982) = 52.46, p < .001, r = 0.03). No significant effects were found, after control- ling for channel: "Beginning" (β = −1.28, se = were used more often by politically left leaning 1.02, p = .68), "Middle" (β = −1.65, se = news channels. −1.65, p = .11), "End" (β = −1.79, se = 1.09, p = .1). 4.4 Sentiment clusters and popularity In addition we used an OLS regression model To assess the relationship between the sentiment to test if the average sentiment score of each tran- clusters and political orientations on popularity script and political leaning had an effect on the rating, we conducted three least square regres- adjusted popularity (F (3, 15989) = 1759, p < sion models in R with the "caret" package (Kuhn, .001, r2 = 0.248). It seems that the model can 2008). The different sentiment clusters were the account for 24% of the variance in adjusted pop- predictors for the adjusted popularity rating. We ularity. The model exhibits a significant con- used the cluster "mood swing" as the reference stant (β = 0.013, se < 0.001, p < 0.001), a category as it was closest to the overall average significant main effect of political leaning (right) of the adjusted upvotes, all other clusters were (β = 0.021, se < 0.001, p < 0.001), an in- treated as a separate dichotomous variable. Our significant main effect for average sentiment (β = first regression model included sentiment clusters, −0.0004, se = 0.001, p = 0.66), and a significant which consisted of all news channels. The second interaction between average sentiment and politi- regression model included the channel as a fixed cal leaning (right) (β = −0.003, se < 0.001, p = effect along with sentiment clusters. The analysis 0.034). The model predicted adjusted popular- indicated that there was no significant difference ity for left wing channel when average sentiment in the adjusted popularity rating between the clus- equals to zero is 0.013 and 0.013 + 0.02 = 0.033 ters in the second model; "Rags to riches" (β = for right wing channels. The slope of the regres- −1.28, se = 1.60, p = .42), "Riches to rags" (β = sion line for the left wing channel is -0.0004 and .47, se = 1.61, p = .77), "Down hill from here" -0.0004 - 0.003 = -0.0034 for right wing channels, (β = −1.25, se = 1.69, p = .46), "End on a high suggesting that the effect of average sentiment is note" (β = .4, se = 1.69, p = .81), "Up hill from greater in magnitude for right wing than for left her" (β = −1.71, se = 1.71, p = .32), "End on wing channels.

89 5 Discussion et al., 2019b)). Further research could examine the distribution In this study we examined sentiment patterns in of sentiment patterns between specific YouTube news videos published on YouTube channels with channels and investigate how factors, such as different political leanings. Using sentiment tra- viewership or content influence them. This could jectory analysis, we identified recurring patterns include how pro-right/left channels differ in the of sentiment changes, which were then grouped amount of content creation for specific topics and through k-means clustering into seven major cat- how sentiment is used within this content. In addi- egories of videos based on their sentiment pat- tion, future work could examine if political lean- terns (e.g. "rags to riches" or "mood swings"). ing has predictive value on sentiment clusters for These patterns correspond to the ones identified in specific topics. This could be useful to ascertain previous research on sentiment patterns of vlog- whether news channels with a right or left politi- gers on YouTube (Kleinberg et al., 2018). While cal leaning discuss specific topics more often and YouTube videos of vloggers and news channels differently than others and with what type of sen- differ in their domain, the persistence of similar timent style. sentiment clusters might indicate the presence of a few consistent sentiment styles that are shared be- 5.2 Predicting video popularity through tween specific content domains and themes across sentiment patterns YouTube. Future research could further examine Our study indicated that sentiment patterns are whether similar patterns persist across various do- weak predictors of news video popularity. A mains of YouTube content. number of studies suggest that content-based features, in particular sentiment, have a strong impact 5.1 Sentiment pattern by political stance on news content popularity (Trussler and Soroka, Our results show, that political leanings seem to 2014; Soroka et al., 2015). However, our findings influence the usage of sentiment patterns: "Down- align with results of other studies that emphasize hill from here" was used more often by pro-right the importance of looking at a broader set of fea- news channels than by pro-left news channels, and tures, in particular contextual ones (e.g., the time conversely "Uphill from here" was used more of- of publication), for predicting the popularity of ten by pro-left news channels. It is interesting news content (Tatar et al., 2012; Keneshloo et al., that both sentiment patterns exhibit the same pro- 2016). Additionally, in the case of YouTube, the portion of negative and positive sentiment (80/20 importance of other content-agnostic factors (e.g., respectively), but are different in sentiment order the total views a channel received previously) for (see figure1). predicting the popularity of videos has been noted It is important to note that previous studies show (Borghol et al., 2012; Figueiredo et al., 2014). that the sentiment of politics-related content, such It is important to note that the current prediction as political ads, can affect the emotional state of task, with sentiment clusters, does not account for the viewer (Brader, 2005). For the "Downhill from the temporal properties of the sentiment trajecto- here" and "Uphill from here" patterns, viewers of ries. Future research could utilize the sequential pro-right news channels are left with a more nega- alignment of the raw sentiment scores by integrat- tive sentiment and viewers of pro-left news chan- ing this aspect into a prediction task of video pop- nels are left with a more positive sentiment after ularity. That way, the features could be used with- watching the whole video. out condensing information (e.g., into clusters) At the same time, while some sentiment pat- and acknowledge the sequential nature of senti- terns seem to be more commonly used by channels ment trajectories. with specific political leaning, it is difficult to propose a strong theoretical background which can 5.3 Predicting popularity through average explain this link. The complexity of this task is re- sentiment and political stance lated to the large number of factors (both ideolog- We also tested for effects of average sentiment ical, but also contextual) which can influence the scores and political leaning on popularity with. use of language for political purposes (for a more The regression model was able to account for 24% detailed discussion see recent work on linguis- of the variance of video popularity. Examining tic complexity of political speeches (Schoonvelde the model’s coefficients more closely show, that

90 popularity seem to be higher when the video origi- We found seven sentiment shapes similar to those nated from a pro-right news channel. Furthermore, found in previous research. The cluster "Mood the interaction of political leaning and sentiment swings" was most prominent whereas "End on a scores shows, that a negative sentiment will in- low note" was least prominent. Two additional crease popularity, while a positive sentiment will sentiment clusters seemed to be used differently decrease popularity more for videos from pro- depending on political leaning of the channels: the right news channels. Our findings support earlier cluster "Downhill from here" was used more often work, which show that a negative tone seem to be by pro-right news channels than by pro-left news more popular overall (Trussler and Soroka, 2014; channels. The reversed effect was observed for Soroka et al., 2015). the cluster "Uphill from here". In addition, sentiment clusters seem to have no predictive value on 5.4 Limitations popularity ratings. However, we found that pro- The current dataset consists of transcripts of right videos were more popular and that negative YouTube videos and it is important to recognize sentiments increased popularity, for averaged sen- that aspects such as video and audio of the clips timent scores of each video. Future research on are not integrated in the analyses. News channels dynamic approaches to sentiment analysis might might utilize audio and visual effects differently, help overcome some of the current limitations and which could affect text sentiment and video popu- offer more nuanced insights into language use in larity. online media. In addition, the obtained transcripts in our analyses could have been generated manually or au- References tomatically, hence might differ in quality. Since there is no direct indicator of this, we do not know Frank Bentley, Katie Quehl, Jordan Wirfs-Brock, and in what proportion they are represented in our cor- Melissa Bica. 2019. Understanding Online News Behaviors. page 11. pus. Generating appropriate and accurate bias rat- Youmna Borghol, Sebastien Ardon, Niklas Carlsson, ings for news channels is not easy. Therefore, Derek Eager, and Anirban Mahanti. 2012. The un- told story of the clones: Content-agnostic factors it is not guaranteed that the bias rating of Media that impact YouTube video popularity. In Proceed- Bias/Fact Check is accurate in all regards. How- ings of the 18th ACM SIGKDD International Con- ever, it has a comprehensive list of news channels, ference on Knowledge Discovery and Data Mining - which are not always covered by other bias rating KDD ’12, page 1186, Beijing, China. ACM Press. resources (Budak et al., 2016; Center, 2018). Alexandre Bovet and Hernán A. Makse. 2019. In- Finally, in our study we specifically focused on fluence of fake news in Twitter during the 2016 YouTube videos. While earlier studies (Peer and US presidential election. Nature Communications, 10(1). Ksiazek, 2011; al Nashmi et al., 2017) demon- strate that content produced by legacy media for Roberta Bracciale and Antonio Martella. 2017. Define YouTube often follows the same standards and the populist political communication style: The case formats as stories produced for other platforms, of Italian political leaders on Twitter. Information, Communication & Society, 20(9):1310–1329. there also exceptions from this rule. Some news organizations (for instance, RT) tend to push Ted Brader. 2005. Striking a Responsive Chord: How more provocative stories to YouTube, whereas Political Ads Motivate and Persuade Voters by Ap- pealing to Emotions. American Journal of Political others (such as CNN) preferred to publish more Science, 49(2):388. lighter content on the platform (al Nashmi et al., 2017). These distinct features of content dis- Ceren Budak, Sharad Goel, and Justin M. Rao. 2016. tributed through YouTube news channels can im- Fair and Balanced? Quantifying Media Bias through Crowdsourced Content Analysis. Public Opinion pact our observations. Quarterly, 80(S1):250–271. 6 Conclusion E. Cambria. 2016. Affective Computing and Sentiment Analysis. IEEE Intelligent Systems, 31(2):102–107. In this study we showed that news channels Erik Cambria, Jie Fu, and Federica Bisio. 2015. Af- on YouTube exhibit different sentiment patterns, fectiveSpace 2: Enabling Affective Intuition for which can be clustered into overarching groups. Concept-Level Sentiment Analysis. page 7.

91 Pew Research Center. 2018. Trends and Facts on On- Michael Hameleers, Linda Bos, and Claes H. de line News | State of the News Media. Vreese. 2017.“ They Did It”: The Effects of Emo- tionalized Blame Attribution in Populist Communi- Mark Coddington. 2015. Clarifying Journalism’s cation. Communication Research, 44(6):870–900. Quantitative Turn: A typology for evaluating data journalism, computational journalism, and Natali Helberger. 2015. Merely Facilitating or Actively computer-assisted reporting. Digital Journalism, Stimulating Diverse Media Choices? Public Service 3(3):331–348. Media at the Crossroad. page 17.

Claes H de Vreese. 2005. News framing: Theory and Matthew Jockers. 2015. » Revealing Sentiment and typology. page 12. Plot Arcs with the Syuzhet Package Matthew L. Jockers. Devin M. Dotson, Susan K. Jacobson, Lynda Lee Kaid, and J. Stuart Carlton. 2012. Media Coverage of Cli- Mesut Kaya, Guven Fidan, and Ismail H. Toroslu. mate Change in Chile: A Content Analysis of Con- 2012. Sentiment Analysis of Turkish Political servative and Liberal Newspapers. Environmental News. In 2012 IEEE/WIC/ACM International Con- Communication, 6(1):64–81. ferences on Web Intelligence and Intelligent Agent Technology, pages 174–180, Macau, China. IEEE. Sven Engesser, Nicole Ernst, Frank Esser, and Florin Büchel. 2017. Populism and social media: How Yaser Keneshloo, Shuguang Wang, Eui-Hong (Sam) politicians spread a fragmented ideology. Informa- Han, and Naren Ramakrishnan. 2016. Predicting the tion, Communication & Society, 20(8):1109–1126. Popularity of News Articles. In Proceedings of the 2016 SIAM International Conference on Data Min- Sarah Eskens, Natali Helberger, and Judith Moeller. ing, pages 441–449. Society for Industrial and Ap- 2017. Challenged by news personalisation: Five plied Mathematics. perspectives on the right to receive information. Journal of Media Law, 9(2):259–284. Bennett Kleinberg, Maximilian Mozes, and Isabelle van der Vegt. 2018. Identifying the sentiment styles Lauren Feldman, Edward W. Maibach, Connie Roser- of YouTube’s vloggers. In Proceedings of the 2018 Renouf, and Anthony Leiserowitz. 2012. Climate Conference on Empirical Methods in Natural Lan- on Cable: The Nature and Impact of Global Warm- guage Processing, page 10. ing Coverage on Fox News, CNN, and MSNBC. The International Journal of Press/Politics, 17(1):3– Max Kuhn. 2008. Building Predictive Models in R Us- 31. ing the caret Package. Journal of Statistical Soft- ware, 28(5). Flavio Figueiredo, Jussara M Almeida, Marcos An- dre Gonc Alves, and Fabricio Benevenuto. 2014. Bing Liu. 2010. Sentiment Analysis and Subjectivity. On the Dynamics of Social Media Popularity: A page 38. YouTube Case Study. ACM Transactions on Inter- net Technology, 1(1):22. Mika V. Mäntylä, Daniel Graziotin, and Miikka Kuu- tila. 2018. The evolution of sentiment analysis—A Seth Flaxman, Sharad Goel, and Justin M. Rao. review of research topics, venues, and top cited pa- 2016. Filter Bubbles, Echo Chambers, and On- pers. Computer Science Review, 27:16–32. line News Consumption. Public Opinion Quarterly, 80(S1):298–320. Maxwell E. McCombs. 2014. Setting the Agenda: The Mass Media and Public Opinion, 2. ed edition. Tomohiro Fukuhara, Hiroshi Nakagawa, and Toyoaki Polity Press, Cambridge. OCLC: 935284613. Nishida. 2007. Understanding Sentiment of People from News Articles: Temporal Sentiment Analysis Rohit Mehta and Lynette DeAun Guzmán. 2018. Fake of Social Events. page 2. or Visual Trickery? Understanding the Quantitative Visual Rhetoric in the News. Journal of Media Lit- J. Gao, M. L. Jockers, J. Laudun, and T. Tangher- eracy Education, page 19. lini. 2016. A multiscale theory for the dynamical evolution of sentiment in novels. In 2016 Inter- Eisa al Nashmi, Michael North, Terry Bloom, and Jo- national Conference on Behavioral, Economic and hanna Cleary. 2017. Promoting a global brand: Socio-Cultural Computing (BESC), pages 1–4. a study of international news organisationsâA˘ Z´ youtube channels. The Journal of International Matthew Gentzkow and Jesse M. Shapiro. 2011. Ideo- Communication, 23(2):165–185. logical Segregation Online and Ofﬂine *. The Quar- terly Journal of Economics, 126(4):1799–1839. N. Newman, R. Fletcher, A. Kalogeropoulos, D. Levy, and R. K. Nielsen. 2018. Reuters Institute Digital Jae Sik Ha and Donghee Shin. 2016. Framing the News Report. http://www.digitalnewsreport.org/. Arab Spring: Partisanship in the news stories of Korean Newspapers. International Communication Bo Pang and Lillian Lee. 2008. Opinion mining and Gazette, 78(6):536–556. sentiment analysis. page 94.

92 Limor Peer and Thomas B Ksiazek. 2011. Youtube and Robert L. Thorndike. 1953. Who belongs in the fam- the challenge to journalism: new standards for news ily? Psychometrika, 18(4):267–276. videos online. Journalism Studies, 12(1):45–63. Marc Trussler and Stuart Soroka. 2014. Consumer De- Leonard Richardson. 2019. Beautiful Soup. mand for Cynical and Negative News Frames. The International Journal of Press/Politics, 19(3):360– Tyler Rinker. 2019a. A data package contain- 379. ing lexicons and dictionaries for text analysis: Trinker/lexicon. Peter Van Aelst, Jesper Strömbäck, Toril Aalberg, Frank Esser, Claes de Vreese, Jörg Matthes, David Tyler Rinker. 2019b. Dictionary based senti- Hopmann, Susana Salgado, Nicolas Hubé, Ag- ment analysis that considers valence shifters: nieszka St˛epinska,´ Stylianos Papathanassopoulos, Trinker/sentimentr. Rosa Berganza, Guido Legnante, Carsten Reine- mann, Tamir Sheafer, and James Stanyer. 2017. Po- Martijn Schoonvelde, Anna Brosius, Gijs Schumacher, litical communication in a high-choice media envi- and Bert N. Bakker. 2019a. Liberals lecture, con- ronment: A challenge for democracy? Annals of the servatives communicate: Analyzing complexity and International Communication Association, 41(1):3– ideology in 381,609 political speeches. PLOS ONE, 27. 14(2):e0208450. Tyler Welch. 2018. Theology, heroism, justice, and Martijn Schoonvelde, Anna Brosius, Gijs Schumacher, fear: An analysis of ISIS propaganda magazines and Bert N Bakker. 2019b. Liberals lecture, con- Dabiq and Rumiyah. Dynamics of Asymmetric Con- servatives communicate: Analyzing complexity and ﬂict, 11(3):186–198. ideology in 381,609 political speeches. PloS one, 14(2):e0208450. Lori Young and Stuart Soroka. 2012. Affective News: The Automated Coding of Sentiment in Political Kim Christian Schrøder. 2015. News Media Old Texts. Political Communication, 29(2):205–231. and New: Fluctuating audiences, news repertoires and locations of consumption. Journalism Studies, 16(1):60–78.

Saif Shahin, Pei Zheng, Heloisa Aruth Sturm, and Deepa Fadnis. 2016. Protesting the Paradigm: A Comparative Study of News Coverage of Protests in Brazil, China, and India. The International Journal of Press/Politics, 21(2):143–164.

Susan Currie Sivek. 2018. Both Facts and Feelings: Emotion and News Literacy. Journal of Media Lit- eracy Education, page 16.

Stuart Soroka, Lori Young, and Meital Balmas. 2015. Bad News or Mad News? Sentiment Scoring of Negativity, Fear, and Anger in News Content. The ANNALS of the American Academy of Political and Social Science, 659(1):108–121.

Cass R. Sunstein. 2017. #Republic: Divided Democ- racy in the Age of Social Media. Princeton Univer- sity Press, Princeton ; Oxford.

M. Iftekhar Tanveer, Samiha Samrose, Raiyan Abdul Baten, and M. Ehsan Hoque. 2018. Awe the Audi- ence: How the Narrative Trajectories Affect Audi- ence Perception in Public Speaking. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’18, pages 1–12, Mon- treal QC, Canada. ACM Press.

A. Tatar, P. Antoniadis, M. D. de Amorim, and S. Fdida. 2012. Ranking News Articles Based on Popularity Prediction. In 2012 IEEE/ACM Inter- national Conference on Advances in Social Net- works Analysis and Mining, pages 106–110, Istan- bul. IEEE.