Decoupling Search Topic and Popularity Ranking: Insights on Online Echo Chambers
Total Page:16
File Type:pdf, Size:1020Kb
MIT IDE RESEARCH BRIEF VOL. 2017.8 DECOUPLING SEARCH TOPIC AND POPULARITY RANKING: INSIGHTS ON ONLINE ECHO CHAMBERS Sagit Bar-Gill and Neil Gandal Just as sound reverberates in an enclosed space, so do online environment for content exploration, we broke down opinions and information “bounce” among social circles. search into two dimensions — content topic and content Since “birds of a feather flock together,” these social popularity. We then looked at the ways individuals moved circles consist mostly of like-minded individuals, making through the material. We observed, for instance, the weight it harder for dissenting views to permeate our discourse, that users placed on their own interests versus those of and creating an echo of our own information and views the crowd, and how these patterns relate to an individual’s (1). This echo chamber phenomenon has drawn growing characteristics, such as how social they regard themselves concern in post-election USA, post-Brexit UK, and pretty and whether they try to influence others in their social much everywhere in our increasingly connected world (2– circles (what we term “opinion leaders”). 4). Echo chambers have always been around as a natural result of our homophilous social choices. But as our social In our experimental environment, susceptibility to echo networking continues its shift online, the echoes seem to be chambers is well-proxied by: (I) conducting relatively getting worse (2, 4, 5). little exploration in the search process, and (II) relying on popularity in content choice. Who is responsible for our online echo chambers – ourselves or filtering and ranking algorithms beyond our Therefore, users who conduct little exploration and rely control? Filtering and ranking algorithms are implemented more heavily on the crowd’s previous choices will, over in many of our online environments, creating much- time, see less diverse content. As a result, they’ll be at a debated personalization of search results (6–9), and higher risk of getting caught in an echo chamber. holding the potential to influence individual behaviors and aggregate outcomes (10). Yet the human factor carries substantial weight, too. IN THIS RESEARCH BRIEF Bakshy and colleagues at Facebook research labs (11) find that individuals’ own choices are more responsible for • Echo chambers have always been around as a natural result of homophilous social choices. But their echo chambers than Facebook’s newsfeed ranking as social networking continues its shift online, the algorithm. In line with this, recent Pew research (12) finds echoes seem to be getting worse. that 83% of social media users ignore political posts with • Algorithmic ranking, based on popularity, users’ which they disagree. previous searches, clicks, and other factors, is an integral part of search engines, social media, and We primarily examined two factors: The types of users other online environments. • Ranking of results has been shown to have a most likely to get caught in content echo chambers, and major influence on user choices, and could intro- the role of displaying popularity information (view counts), duce biases that go unnoticed as users are large- in facilitating content echo chambers. By creating a simple ly unaware of the mechanisms determining their search results and news feeds. • Highly social users, those searching in a content space they are already familiar with, and young users, are all at a higher risk of echo chambers. • Users who consistently search by topics of inter- est and rely less on the crowd’s previous choices were less likely to suffer from echo chambers. MIT IDE RESEARCH BRIEF VOL. 2017.8 DECOUPLING SEARCH TOPIC AND POPULARITY RANKING Sagit Bar-Gill and Neil Gandal Opinion leaders – individuals who influence others’ opinions or choices (13–20) – are likely to affect their peers’ content exposure. We thus specifically studied their search patterns, comparing influencers to individuals who are not opinion leaders. THE EXPERIMENTAL SETTING In our experimental search environment, named TED- it, 1,846 study participants explored the collection of TED talks posted on YouTube (roughly 1,600 short videos).1 Participants navigated using two buttons, Category and Popularity. The Category button allowed users to choose one of 15 content groupings and presented a list of talks in random order, without any ranking. By contrast, the Popularity button sorted the displayed search results by their Figure 1: A screenshot with search results on TED-it number of views on YouTube — from most to least popular as it appeared after a user clicked on Category, and — or simply sorted all talks by popularity if no category was chose the Entertainment category. chosen. Users could click each of the buttons as many times as they like, creating a search sequence. This search journey was the object of our study. The relationship between search patterns and viewers’ who relied strongly on Popularity sorting and conducted social characteristics was further determined by a series of little to no Category search, were more susceptible to questionnaires, which assessed users’ sociability, opinion echo chambers in our research setting. leadership, and previous experience with TED content, along with some demographics. USER CHARACTERISTICS AND EXPLORATION PATTERNS As with most content platforms, there are popular TED talks as well as many others with fewer views. We examined how user characteristics such as sociability, We found that generally, people who explored content familiarity with content, opinion leadership and age affect with less reliance on its popularity metrics ended up content exploration. We operationalized “exploration” by at content that is less known by their peers, but more studying users’ search flow in our environment: from their first suited to their personal interests. As assumed, users click, to follow-up clicks, to the type of results they choose, who consistently search by topics of interest and rely and the chosen talk’s location (or, scroll depth) in that list. less on the crowd’s previous choices were less likely to suffer from echo chambers. On the flip side, people Three types of people were most at risk for falling into echo chambers: highly social individuals; those already familiar 1 TED is a nonprofit organization devoted to spreading ideas, with TED content, and young users. Each was more likely usually in the form of short talks (18 minutes or less). TED to rely on popularity considerations and explore less. stands for Technology, Entertainment and Design, though TED talks today may cover any topic (more at https://www.ted.com/ about/). We use TED talks in compliance with their Creative Commons license. IDE.MIT.EDU 2 MIT IDE RESEARCH BRIEF VOL. 2017.8 DECOUPLING SEARCH TOPIC AND POPULARITY RANKING Sagit Bar-Gill and Neil Gandal Additionally, opinion leaders were more likely to explore the already familiar with, and young users, are all at a videos and had lower reliance on popularity sorting. We higher risk of such echo chambers. found that male opinion leaders in our sample conducted more topic-based exploration and invested more effort in Opinion leaders may ameliorate echo chamber concerns search than male non-leaders. Apparently, these opinion within their social circles, due to their tendency to conduct leaders are more likely to seek new avenues for influence, more popularity-independent exploration compared to and look for novel content to introduce to their followers that their followers. However, their inclination to explore is may ameliorate their group’s own echo chamber.2 highly sensitive to the provision of popularity information, and curtailed by it. We also studied how influenced people are by knowing the popularity of content. With “like” counts and view Should we therefore, suppress or, at least, reduce counts now baked into most online environments, was the visibility of popularity information in our online this information making people more likely or less likely to environments to increase the diversity of content explore unfamiliar territories? consumed? To the extent that content diversity is a desired end, the answer is yes. Furthermore, in our analyses, To answer this question, we randomly assigned users to one there were no statistically significant correlations of two conditions, where popularity information was either between users’ exploration characteristics, and proxies shown or blocked in category-based searches. Interestingly, for enjoyment, such as viewing talks past the mandated in our study, only male opinion leaders were affected by the time, or watching an extra talk. This suggests that, at display of view counts, responding to this information by least in the realm of curated content, such as TED talks, increasing their popularity reliance. popular content is not inherently superior to less popular content. Therefore, designing online environments that Our study was not designed to uncover gender differences encourage exploration (e.g., with increased visibility of in the relationship between user characteristics and content non-hit content, and reduced visibility of view counts) exploration patterns, however, these do exist. Namely, our may alleviate content echo chambers, with little impact regression models and relationships between variables of on user satisfaction. interest are largely statistically significant for males, and not significant for females, indicating the existence of consistent While our experiment takes place in a non-standard patterns for men, but not for women. search environment, it offers first insights as to users’ personal preferences for exploration and to the CONCLUSIONS extent to which they would actively seek out ranked results, if given a choice. Future research may extend Tendencies to conduct limited exploration and rely on these results to more organic settings and general peers’ past choices in own content choice are likely to content spaces. Still, our results may speak to lead users down echo chambers, with limited exposure general concerns raised regarding the growing role to content that is not in line with their peers’ and own of algorithms in our lives 21, 22).