MIT IDE RESEARCH BRIEF VOL. 2017.8
DECOUPLING SEARCH TOPIC AND POPULARITY RANKING: INSIGHTS ON ONLINE ECHO CHAMBERS
Sagit Bar-Gill and Neil Gandal
Just as sound reverberates in an enclosed space, so do online environment for content exploration, we broke down opinions and information “bounce” among social circles. search into two dimensions — content topic and content Since “birds of a feather flock together,” these social popularity. We then looked at the ways individuals moved circles consist mostly of like-minded individuals, making through the material. We observed, for instance, the weight it harder for dissenting views to permeate our discourse, that users placed on their own interests versus those of and creating an echo of our own information and views the crowd, and how these patterns relate to an individual’s (1). This echo chamber phenomenon has drawn growing characteristics, such as how social they regard themselves concern in post-election USA, post-Brexit UK, and pretty and whether they try to influence others in their social much everywhere in our increasingly connected world (2– circles (what we term “opinion leaders”). 4). Echo chambers have always been around as a natural result of our homophilous social choices. But as our social In our experimental environment, susceptibility to echo networking continues its shift online, the echoes seem to be chambers is well-proxied by: (I) conducting relatively getting worse (2, 4, 5). little exploration in the search process, and (II) relying on popularity in content choice. Who is responsible for our online echo chambers – ourselves or filtering and ranking algorithms beyond our Therefore, users who conduct little exploration and rely control? Filtering and ranking algorithms are implemented more heavily on the crowd’s previous choices will, over in many of our online environments, creating much- time, see less diverse content. As a result, they’ll be at a debated personalization of search results (6–9), and higher risk of getting caught in an echo chamber. holding the potential to influence individual behaviors and aggregate outcomes (10).
Yet the human factor carries substantial weight, too. IN THIS RESEARCH BRIEF Bakshy and colleagues at Facebook research labs (11) find that individuals’ own choices are more responsible for • Echo chambers have always been around as a natural result of homophilous social choices. But their echo chambers than Facebook’s newsfeed ranking as social networking continues its shift online, the algorithm. In line with this, recent Pew research (12) finds echoes seem to be getting worse. that 83% of social media users ignore political posts with • Algorithmic ranking, based on popularity, users’ which they disagree. previous searches, clicks, and other factors, is an integral part of search engines, social media, and We primarily examined two factors: The types of users other online environments. • Ranking of results has been shown to have a most likely to get caught in content echo chambers, and major influence on user choices, and could intro- the role of displaying popularity information (view counts), duce biases that go unnoticed as users are large- in facilitating content echo chambers. By creating a simple ly unaware of the mechanisms determining their search results and news feeds. • Highly social users, those searching in a content space they are already familiar with, and young users, are all at a higher risk of echo chambers. • Users who consistently search by topics of inter- est and rely less on the crowd’s previous choices were less likely to suffer from echo chambers. MIT IDE RESEARCH BRIEF VOL. 2017.8 DECOUPLING SEARCH TOPIC AND POPULARITY RANKING
Sagit Bar-Gill and Neil Gandal
Opinion leaders – individuals who influence others’ opinions or choices (13–20) – are likely to affect their peers’ content exposure. We thus specifically studied their search patterns, comparing influencers to individuals who are not opinion leaders.
THE EXPERIMENTAL SETTING
In our experimental search environment, named TED- it, 1,846 study participants explored the collection of TED talks posted on YouTube (roughly 1,600 short videos).1 Participants navigated using two buttons, Category and Popularity. The Category button allowed users to choose one of 15 content groupings and presented a list of talks in random order, without any ranking. By contrast, the Popularity button sorted the displayed search results by their Figure 1: A screenshot with search results on TED-it number of views on YouTube — from most to least popular as it appeared after a user clicked on Category, and — or simply sorted all talks by popularity if no category was chose the Entertainment category. chosen. Users could click each of the buttons as many times as they like, creating a search sequence. This search journey was the object of our study.
The relationship between search patterns and viewers’ who relied strongly on Popularity sorting and conducted social characteristics was further determined by a series of little to no Category search, were more susceptible to questionnaires, which assessed users’ sociability, opinion echo chambers in our research setting. leadership, and previous experience with TED content, along with some demographics. USER CHARACTERISTICS AND EXPLORATION PATTERNS As with most content platforms, there are popular TED talks as well as many others with fewer views. We examined how user characteristics such as sociability, We found that generally, people who explored content familiarity with content, opinion leadership and age affect with less reliance on its popularity metrics ended up content exploration. We operationalized “exploration” by at content that is less known by their peers, but more studying users’ search flow in our environment: from their first suited to their personal interests. As assumed, users click, to follow-up clicks, to the type of results they choose, who consistently search by topics of interest and rely and the chosen talk’s location (or, scroll depth) in that list. less on the crowd’s previous choices were less likely to suffer from echo chambers. On the flip side, people Three types of people were most at risk for falling into echo chambers: highly social individuals; those already familiar 1 TED is a nonprofit organization devoted to spreading ideas, with TED content, and young users. Each was more likely usually in the form of short talks (18 minutes or less). TED to rely on popularity considerations and explore less. stands for Technology, Entertainment and Design, though TED talks today may cover any topic (more at https://www.ted.com/ about/). We use TED talks in compliance with their Creative Commons license.
IDE.MIT.EDU 2 MIT IDE RESEARCH BRIEF VOL. 2017.8 DECOUPLING SEARCH TOPIC AND POPULARITY RANKING
Sagit Bar-Gill and Neil Gandal
Additionally, opinion leaders were more likely to explore the already familiar with, and young users, are all at a videos and had lower reliance on popularity sorting. We higher risk of such echo chambers. found that male opinion leaders in our sample conducted more topic-based exploration and invested more effort in Opinion leaders may ameliorate echo chamber concerns search than male non-leaders. Apparently, these opinion within their social circles, due to their tendency to conduct leaders are more likely to seek new avenues for influence, more popularity-independent exploration compared to and look for novel content to introduce to their followers that their followers. However, their inclination to explore is may ameliorate their group’s own echo chamber.2 highly sensitive to the provision of popularity information, and curtailed by it. We also studied how influenced people are by knowing the popularity of content. With “like” counts and view Should we therefore, suppress or, at least, reduce counts now baked into most online environments, was the visibility of popularity information in our online this information making people more likely or less likely to environments to increase the diversity of content explore unfamiliar territories? consumed? To the extent that content diversity is a desired end, the answer is yes. Furthermore, in our analyses, To answer this question, we randomly assigned users to one there were no statistically significant correlations of two conditions, where popularity information was either between users’ exploration characteristics, and proxies shown or blocked in category-based searches. Interestingly, for enjoyment, such as viewing talks past the mandated in our study, only male opinion leaders were affected by the time, or watching an extra talk. This suggests that, at display of view counts, responding to this information by least in the realm of curated content, such as TED talks, increasing their popularity reliance. popular content is not inherently superior to less popular content. Therefore, designing online environments that Our study was not designed to uncover gender differences encourage exploration (e.g., with increased visibility of in the relationship between user characteristics and content non-hit content, and reduced visibility of view counts) exploration patterns, however, these do exist. Namely, our may alleviate content echo chambers, with little impact regression models and relationships between variables of on user satisfaction. interest are largely statistically significant for males, and not significant for females, indicating the existence of consistent While our experiment takes place in a non-standard patterns for men, but not for women. search environment, it offers first insights as to users’ personal preferences for exploration and to the CONCLUSIONS extent to which they would actively seek out ranked results, if given a choice. Future research may extend Tendencies to conduct limited exploration and rely on these results to more organic settings and general peers’ past choices in own content choice are likely to content spaces. Still, our results may speak to lead users down echo chambers, with limited exposure general concerns raised regarding the growing role to content that is not in line with their peers’ and own of algorithms in our lives 21, 22). views and opinions. Exposure to diverse content may be further limited over time, as user tendencies feed into Algorithmic ranking, based on popularity, users’ previous personalization algorithms. We find that highly social searches, clicks, and other factors, is an integral part users, those searching in a content space they are of search engines, social media, and other online
2 There were no statistically significant patterns for female opin- ion leaders in our sample.
IDE.MIT.EDU 3 MIT IDE RESEARCH BRIEF VOL. 2017.8 DECOUPLING SEARCH TOPIC AND POPULARITY RANKING
Sagit Bar-Gill and Neil Gandal
environments. Ranking of results has been shown to have The null effect we find for the relationship between a major influence on user choices (23), has the potential ranking and user enjoyment, while limited to curated of introducing biases (10), and these impacts easily go content, indicates that the user experience does unnoticed, as users are largely unaware of the black-box not necessarily suffer when ranking is removed. mechanisms determining their search results and news Regulators may use this as support for the case for feeds (10, 24, 25). diversifying search results, at least in domains of national importance, such as news and elections.
References 13. Katz E, Lazarsfeld PF (1955) Personal influence: The Part Played by People in the Flow of Mass Communication (The Free Press, New 1. McPherson M, Smith-Lovin L, Cook JM (2001) Birds of a Feather: York, NY) Available at: https://books.google.com/books?hl=en&lr=&i Homophily in Social Networks. Annu Rev Sociol 27(1):415–444. d=rElW8D0D8gYC&oi=fnd&pg=PR1&dq=Personal+influence:+The+ 2. El-Bermawy MM (2016) Your Echo Chamber is Destroying Democracy | part+played+by+people+in+the+flow+of+mass+communications.&ot WIRED. Wired. Available at: https://www.wired.com/2016/11/filter-bubble- s=ODyZ-Pg_dJ&sig=cmAFXEcFraJWINNm7XfssrELbIM [Accessed destroying-democracy/ [Accessed August 1, 2017]. August 1, 2017]. 3. Chater J (2016) What the EU referendum result teaches us about the 14. Katz E (1957) The Two-Step Flow of Communication: An Up-To- dangers of the echo chamber. New Statesman. Available at: http://www. Date Report on an Hypothesis. Public Opin Q 21(1):61–78. newstatesman.com/2016/07/what-eu-referendum-result-teaches-us- 15. Rogers EM (1995) Diffusion of innovations (Free Press, Glencoe) about-dangers-echo-chamber [Accessed August 1, 2017]. doi:citeulike-article-id:126680. 4. Khalid A (2017) Tech Creates Our Political Echo Chambers. It Might 16. Weimann G (1994) The influentials : people who influence people Also Be A Solution. NPR. Available at: http://www.npr.org/sections/ (State University of New York Press) Available at: https://books. alltechconsidered/2017/04/12/522760479/tech-creates-our-political-echo- google.com/books/about/The_Influentials.html?id=ou5bTZGSOYAC chambers-it-might-also-be-a-solution [Accessed August 1, 2017]. [Accessed August 1, 2017]. 5. Aiello LM, et al. (2012) Friendship prediction and homophily in social 17. Van den Bulte C, Joshi Y V. (2007) New Product Diffusion with media. ACM Trans Web 6(2):1–33. Influentials and Imitators. Mark Sci 26(3):400–421. 6. Personalized https://googleblog.blogspot.com/2009/12/personalized- 18. Flodgren G, et al. (2007) Local opinion leaders: effects on search-for-everyone.html [Accessed August 1, 2017]. professional practice and health care outcomes. Cochrane Database 7. Sullivan D (2011) Bing Results Get Localized & Personalized. of Systematic Reviews, ed Eccles MP (John Wiley & Sons, Ltd, Search Engine L. Available at: http://searchengineland.com/bing-results- Chichester, UK). doi:10.1002/14651858.CD000125.pub3. get-localized-personalized-64284 [Accessed August 1, 2017]. 19. Godes D, Mayzlin D (2009) Firm-Created Word-of-Mouth 8. Pariser E (2011) The Filter Bubble: What the Internet Is Hiding Communication: Evidence from a Field Test. Mark Sci 28(4):721–739. from You (The Penguin Group) Available at: http://dl.acm.org/citation. 20. Iyengar R, Van den Bulte C, Valente TW (2011) Opinion cfm?id=2029079 [Accessed August 1, 2017]. Leadership and Social Contagion in New Product Diffusion. Mark Sci 9. Hannak A, et al. (2013) Measuring personalization of web search. 30(2):195–212. Proceedings of the 22nd International Conference on World Wide Web - 21. Dickey MR (2017) Algorithmic accountability. Tech Crunch. Available WWW ’13 (ACM Press, New York, New York, USA), pp 527–538. at: 20 https://techcrunch.com/2017/04/30/algorithmic-accountability/ 10. Epstein R, Robertson RE (2015) The search engine manipulation [Accessed August 3, 2017]. effect (SEME) and its possible impact on the outcomes of elections. Proc 22. Pasquale F Black box society : the secret algorithms that control Natl Acad Sci U S A 112(33):E4512-21. money and information. 11. Bakshy E, Messing S, Adamic LA (2015) Exposure to 23. Pan B, et al. (2007) In Google We Trust: Users’ Decisions on Rank, ideologically diverse news and opinion on Facebook. Science (80- ) Position, and Relevance. J Comput Commun 12(3):801–823. 348(6239):1130–1132. 24. Schwab P-N (2016) Facebook : how do people perceive NewsFeed 12. Duggan M, Smith A, Page D (2016) The Political Environment on Social curation algorithm. Into Minds. Available at: http://www.intotheminds. Media. Pew Research Center (Washington, D.C.) Available at: http://www. com/blog/en/facebook-how-do-people-perceive-newsfeed-curation- pewinternet.org/2016/10/25/the-political-environment-on-social-media/ algorithm/ [Accessed August 3, 2017]. [Accessed August 1, 2017]. 25. Powers E (2017) New Research Shows Students Are Largely Unaware of News Personalization. Media Shift. Available at: http:// mediashift.org/2017/03/new-research-shows-students-largely- unaware-news-personalization/ [Accessed August 3, 2017].
IDE.MIT.EDU 4 MIT IDE RESEARCH BRIEF VOL. 2017.8 DECOUPLING SEARCH TOPIC AND POPULARITY RANKING
Sagit Bar-Gill and Neil Gandal
Links to this work are online here.
This article is based on the authors’ research project “Content Exploration, Choice and Echo Cham- bers: An Experiment,” which was the subject of CEPR discussion paper no. DP11909 published in March 2017.
Sagit Bar-Gill is a postdoctoral associate at the Neil Gandal is a professor of economics at Tel MIT Sloan School of Management and its Initia- Aviv University and a research fellow at the Cen- tive on the Digital Economy. Details about her tre for Economic Policy Research, in London. work are online at IDE.
MIT INITIATIVE ON THE DIGITAL ECONOMY SUPPORT THE MIT IDE
The MIT IDE is solely focused on the digital economy. The generous support of individuals, foundations, and We conduct groundbreaking research, convene the corporations are critical to the success of the IDE. Their brightest minds, promote dialogue, expand knowledge contributions fuel cutting-edge research by MIT faculty and awareness, and implement solutions that provide and graduate students, and enables new faculty hiring, critical, actionable insight for people, businesses, and curriculum development, events, and fellowships. government. We are solving the most pressing issues Contact Christie Ko ([email protected]) to learn how you of the second machine age, such as defining the future or your organization can support the IDE. of work in this time of unprecendented disruptive digital transformation.
TO LEARN MORE ABOUT THE IDE, INCLUDING UPCOMING EVENTS, VISIT IDE.MIT.EDU
IDE.MIT.EDU 5