DOCSLIB.ORG

You Are How (And Where) You Search? Comparative Analysis of Web Search Behaviour Using Web Tracking Data

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
You Are How (And Where) You Search? Comparative Analysis of Web Search Behaviour Using Web Tracking Data You Are How (and Where) You Search? Comparative Analysis of Web Search Behaviour Using Web Tracking Data Aleksandra Urman,1 2 Mykola Makhortykh, 1 1 University of Bern 2 University of Zurich urman@ifi.uzh.ch, [email protected] Abstract small user samples and can hardly be generalized to broader populations.On the contrary, log-based studies capture the We conduct a comparative analysis of desktop web search behaviour of users from Germany (n=558) and Switzerland behaviour of the large groups of users, but on the aggregate (n=563) based on a combination of web tracking and survey level, thus limiting possibilities for inferring the impact of data. We find that web search accounts for 13% of all desk- users’ individual characteristics on how they search for in- top browsing, with the share being higher in Switzerland than formation. Additionally, log-based studies can not reliably in Germany. We find that in over 50% of cases users clicked infer the connection between search results ranking and user on the first search result, with over 97% of all clicks being behaviour, because researchers can not, in retrospect, iden- made on the first page of search outputs. Most users rely on tify how search results were ranked and presented to individ- Google when conducting searches, and users’ preferences for ual users due to the temporal changes in the results and ef- other engines are related to their demographics. We also test fects of search personalization (Hannak et al. 2013; Kliman- relationships between user demographics and daily number Silver et al. 2015) and randomization (Makhortykh, Urman, of searches, average share of search activities in one’s gen- eral browsing behaviour as well as the tendency to click on and Ulloa 2020; Urman, Makhortykh, and Ulloa 2021). higher- or lower-ranked results. We find differences in such In the present study, we address these limitations by rely- relationships between the two countries that highlights the ing on a type of data source that, to the best of our knowl- importance of comparative research in this domain. Further, edge, has not been used in the context of web search be- we observe differences in the temporal patterns of web search haviour. We utilize the combination of web tracking data use between women and men, marking the necessity of dis- (Christner et al. 2021) with demographic data about individ- aggregating data by gender in observational studies regarding ual users acquired via survey to explore users’ web search online information behaviour. behaviour. Web tracking data includes information on user desktop-based browsing behaviour along with the actual Introduction HTMLs of the browsed content. By acquiring HTMLs of Web search engines are ubiquitous nowadays and act as pages viewed by the users, we can infer the exact composi- major information gate-keepers in high-choice media envi- tion and ranking of web search results users were exposed ronments. Google alone handled around 6.9 billion queries to and, consequently, find out which of these results they per day in 2020 (Petrov 2019) with an average user of clicked on. Google.com turning to the site 18.15 times per day as of Using the combination of web tracking and survey data April 2021 (Alexa 2021). When Google experienced an collected in Germany and Switzerland in spring 2020, we outage in 2013 for 5 minutes, there was a drop of 40% aim to address several gaps in the existing scholarship on in the global web traffic (Svetlik 2013). The numbers are web search behaviour. First, we scrutinize the effect of indi- arXiv:2105.04961v1 [cs.HC] 11 May 2021 staggering, especially given that Google is just one of the vidual demographic characteristics on search behaviour us- search engines - though the dominant one on most mar- ing a large sample of users. Second, unlike earlier large-scale kets. Furthermore, search engines are highly trusted by (i.e., log-based) search behaviour studies, which were fo- their users:according to Edelman Trust Barometer (Edelman cused on single-country populations (usually, the US), our 2021), in 2020 search engines were reported to be the most study offers a comparative perspective and goes beyond the trusted information source globally. US context. Third, we examine the user clicking behavior Given the importance of search engines for shaping pub- in relation to web search results ranking in real-life condi- lic opinion, it is crucial to understand users’ web search be- tions - in contrast to eye-tracking studies that typically rely haviours. Yet, our knowledge in this context remains limited on smaller samples and are carried out in lab settings. and primarily relies on two types of data. eye-tracking (Pan Specifically, we address the following research questions: et al. 2007; Schultheiß, Sunkler,¨ and Lewandowski 2018) 1) how frequently do users with different demographic char- and search engine transaction log data (Jansen and Spink acteristics and socio-economic status use search engines?; 2006; Weber and Jaimes 2011). Both of these data sources 2) what are the temporal patterns of web search use and have their limitations: eye-tracking studies typically rely on do they differ by demographics? 3) are there demographic or socio-economic status-based differences in the choice of in short queries and rarely navigate beyond page 1 of the specific search engines (i.e., Google/Bing/other)?; 4) how search engine. Similar findings were reported by authors of does the rank of a search result relate to the clicking be- a 1-week-long study based on a Korean search engine Naver haviour of users with different demographics? We also ex- (Park, Ho Lee, and Jin Bae 2005). amine country-level differences in relation to each of the Log data has also been utilized to examine temporal as- four questions. pects of web search (Zhang, Jansen, and Spink 2009) and the patters of search query usage (Weber and Jaimes 2011). Related work Such studies allow inferring real-life web search usage pat- Studies on web search behaviour to date have relied on either terns and are based on large data samples - as contrasted of the two data source types: eye-tracking and search engine to eye-tracking-based lab studies. However, log-based stud- transaction log data. ies also have several limitations. First, due to the difficulty Eye-tracking-based studies are typically conducted on of obtaining search logs data owned by proprietary compa- smaller samples, usually not demographically representative nies, most of the transaction logs-based studies focus on sin- ones, and within lab settings. The advantage of such stud- gle search engines. It underminesthe generalizability of their ies is that they allow examining user attention patterns in findings since usage patterns ocan be affected by the dif- the context of web search and, for instance, exploring the ferences in search engine interfaces and/or the differences relation between the ranking of search results and users’ in the demographics of their users. Even studies such as clicking behaviours. In one of the earliest studies (Granka, (Jansen and Spink 2006) that analyze log data from multiple Joachims, and Gay 2004), the authors have examined atten- search engines can not match the users across these engines, tion and clicking patterns of web search engine users based which prevents them from examining ifthe same users uti- on a student sample (n=36), and found that top-ranked re- lize multiple different engines and, if so, whether and how sults receive disproportionately more attention and clicks their behavior is different depending on the engine. than lower-ranked ones. This finding was corroborated in The absence of reliable demographic data about the users numerous further studies (e.g., (Pan et al. 2007; Schultheiß, is another limitation of logs-based studies. Such data is Sunkler,¨ and Lewandowski 2018; Joachims et al. 2007; sometimes available on users’ gender and age, but not on Guan and Cutrell 2007)). other variablessuch as education or income level that can Eye-tracking studies have also investigated the impact of only be inferred by the researchers (e.g., (Weber and Jaimes additional factors on search result selection. For instance, 2011)). However, even such inference-based studies are rare two studies used a small (n=18) sample of users of di- Third, log-based data is inherently noisy, because search re- verse ages and occupations (Joachims et al. 2007) and a quests might be executed not only by human users but also student sample (n=22) (Pan et al. 2007) from the US found by bots, and it is difficult to differentiate between organic that clicking decisions are influenced not only by ranking and automated requests (Jiang, Pei, and Li 2013). Finally, but also perceived relevance of search results. A replication transaction log data, does not allow tracing the position of of the latter study (Schultheiß, Sunkler,¨ and Lewandowski the search results a user clicked on and the only ranking- 2018) conducted circa 10 years after the original one on a related parameter availableis the number of the search result student sample (n=28) in Germany has found similar effects page on which a user selected a result. thus indicating the stability of observed effects across time The aforementioned limitations of both approaches can and different national contexts. be addressed by utilizing web tracking data that includes Despite providing important insights in user search be- full HTMLs of the pages browsed by users and is combined haviour, eye-tracking studies are subjected to a number with survey data. Unlike eye-tracking, this approach is scal- of limitations, in particular their limited scalability. While able and allows observing user behavior in real-life circum- some potential solutions for scaling are being offered in re- stances, not in a lab setting.
Load more

Recommended publications
  • Duckduckgo Search Engines Android
    Duckduckgo search engines android Continue 1 5.65.0 10.8MB DuckduckGo Privacy Browser 1 5.64.0 10.8MB DuckduckGo Privacy Browser 1 5.63.1 10.78MB DuckduckGo Privacy Browser 1 5.62.0 10.36MB DuckduckGo Privacy Browser 1 5.61.2 10.36MB DuckduckGo Privacy Browser 1 5.60.0 10.35MB DuckduckGo Privacy Browser 1 5.59.1 10.35MB DuckduckGo Privacy Browser 1 5.58.1 10.33MB DuckduckGo Privacy Browser 1 5.57.1 10.31MB DuckduckGo Privacy browser © DuckduckGo. Privacy, simplified. This article is about the search engine. For children's play, see duck, duck, goose. Internet search engine DuckDuckGoScreenshot home page DuckDuckGo on 2018Type search engine siteWeb Unavailable inMultilingualHeadquarters20 Paoli PikePaoli, Pennsylvania, USA Area servedWorldwideOwnerDuck Duck Go, Inc., createdGabriel WeinbergURLduckduckgo.comAlexa rank 158 (October 2020 update) CommercialRegregedSeptember 25, 2008; 12 years ago (2008-09-25) was an Internet search engine that emphasized the privacy of search engines and avoided the filter bubble of personalized search results. DuckDuckGo differs from other search engines by not profiling its users and showing all users the same search results for this search term. The company is based in Paoli, Pennsylvania, in Greater Philadelphia and has 111 employees since October 2020. The name of the company is a reference to the children's game duck, duck, goose. The results of the DuckDuckGo Survey are a compilation of more than 400 sources, including Yahoo! Search BOSS, Wolfram Alpha, Bing, Yandex, own web scanner (DuckDuckBot) and others. It also uses data from crowdsourcing sites, including Wikipedia, to fill in the knowledge panel boxes to the right of the results.
    [Show full text]
  • EFF, Duckduckgo, and Internet Archive Amicus Brief
    Case: 17-16783, 11/27/2017, ID: 10667942, DktEntry: 42, Page 1 of 39 NO. 17-16783 IN THE UNITED STATES COURT OF APPEALS FOR THE NINTH CIRCUIT HIQ LABS, INC., PLAINTIFF-APPELLEE, V. LINKEDIN CORPORATION, DEFENDANT-APPELLANT. On Appeal from the United States District Court for the Northern District of California Case No. 3:17-cv-03301-EMC The Honorable Edward M. Chen, District Court Judge BRIEF OF AMICI CURIAE ELECTRONIC FRONTIER FOUNDATION, DUCKDUCKGO, AND INTERNET ARCHIVE IN SUPPORT OF PLAINTIFF-APPELLEE Jamie Williams Corynne McSherry Cindy Cohn Nathan Cardozo ELECTRONIC FRONTIER FOUNDATION 815 Eddy Street San Francisco, CA 94109 Email: [email protected] Telephone: (415) 436-9333 Counsel for Amici Curiae Case: 17-16783, 11/27/2017, ID: 10667942, DktEntry: 42, Page 2 of 39 DISCLOSURE OF CORPORATE AFFILIATIONS AND OTHER ENTITIES WITH A DIRECT FINANCIAL INTEREST IN LITIGATION Pursuant to Rule 26.1 of the Federal Rules of Appellate Procedure, Amici Curiae Electronic Frontier Foundation, DuckDuckGo, and Internet Archive each individually state that they do not have a parent corporation and that no publicly held corporation owns 10 percent or more of their stock. i Case: 17-16783, 11/27/2017, ID: 10667942, DktEntry: 42, Page 3 of 39 TABLE OF CONTENTS CORPORATE DISCLOSURE STATEMENTS ..................................................... i TABLE OF CONTENTS ...................................................................................... ii TABLE OF AUTHORITIES ...............................................................................
    [Show full text]
  • The Autonomous Surfer
    Renée Ridgway The Autonomous Surfer CAIS Report Fellowship Mai bis Oktober 2018 GEFÖRDERT DURCH RIDGWAY The Autonomous Surfer Research Questions The Autonomous Surfer endeavoured to discover the unknown unknowns of alternative search through the following research questions: What are the alternatives to Google search? What are their hidden revenue models, even if they do not collect user data? How do they deliver divergent (and qualitative) results or knowledge? What are the criteria that determine ranking and relevance? How do p2p search engines such as YaCy work? Does it deliver alternative results compared to other search engines? Is there still a movement for a larger, public index? Can there be serendipitous search, which is the ability to come across books, articles, images, information, objects, and so forth, by chance? Aims and Projected Results My PhD research investigates Google search – its early development, its technological innovation, its business model of the past 20 years and how it works now. Furthermore, I have experimented with Tor (The Onion Router) in order to find out if I could be anonymous online, and if so, would I receive diver- gent results from Google with the same keywords. For my fellowship at CAIS I decided to first research search engines that were incorporated into the Tor browser as default (Startpage, Disconnect) or are the default browser now (DuckDuckGo). I then researched search engines in my original CAIS proposal that I had come across in my PhD but hadn’t had the time to research; some are from the Society of the Query Reader (2014) and others I found en route or on colleagues’ suggestions.
    [Show full text]
  • GOGGLES: Democracy Dies in Darkness, and So Does the Web
    GOGGLES: Democracy dies in darkness, and so does the Web Brave Search Team Brave Munich, Germany [email protected] Abstract of information has led to a significant transfer of power from cre- This paper proposes an open and collaborative system by which ators to aggregators. Access to information has been monopolized a community, or a single user, can create sets of rules and filters, by companies like Google and Facebook [27]. While everything is called Goggles, to define the space which a search engine can pull theoretically still retrievable, in practice we are looking at the world results from. Instead of a single ranking algorithm, we could have through the biases of a few providers, who act, unintentionally or as many as needed, overcoming the biases that a single actor (the not, as gatekeepers. Akin to the thought experiment about the tree search engine) embeds into the results. Transparency and openness, falling in the forest [3], if a page is not listed on Google’s results all desirable qualities, will become accessible through the deep re- page or in the Facebook feed, does it really exist? ranking capabilities Goggles would enable. Such system would be The biases of Google and Facebook, whether algorithmic, data made possible by the availability of a host search engine, providing induced, commercial or political dictate what version of the world the index and infrastructure, which are unlikely to be replicated we get to see. Reality becomes what the models we are fed depict without major development and infrastructure costs. Besides the it to be [24].
    [Show full text]
  • Mass Surveillance
    Mass Surveillance Mass Surveillance What are the risks for the citizens and the opportunities for the European Information Society? What are the possible mitigation strategies? Part 1 - Risks and opportunities raised by the current generation of network services and applications Study IP/G/STOA/FWC-2013-1/LOT 9/C5/SC1 January 2015 PE 527.409 STOA - Science and Technology Options Assessment The STOA project “Mass Surveillance Part 1 – Risks, Opportunities and Mitigation Strategies” was carried out by TECNALIA Research and Investigation in Spain. AUTHORS Arkaitz Gamino Garcia Concepción Cortes Velasco Eider Iturbe Zamalloa Erkuden Rios Velasco Iñaki Eguía Elejabarrieta Javier Herrera Lotero Jason Mansell (Linguistic Review) José Javier Larrañeta Ibañez Stefan Schuster (Editor) The authors acknowledge and would like to thank the following experts for their contributions to this report: Prof. Nigel Smart, University of Bristol; Matteo E. Bonfanti PhD, Research Fellow in International Law and Security, Scuola Superiore Sant’Anna Pisa; Prof. Fred Piper, University of London; Caspar Bowden, independent privacy researcher; Maria Pilar Torres Bruna, Head of Cybersecurity, Everis Aerospace, Defense and Security; Prof. Kenny Paterson, University of London; Agustín Martin and Luis Hernández Encinas, Tenured Scientists, Department of Information Processing and Cryptography (Cryptology and Information Security Group), CSIC; Alessandro Zanasi, Zanasi & Partners; Fernando Acero, Expert on Open Source Software; Luigi Coppolino,Università degli Studi di Napoli; Marcello Antonucci, EZNESS srl; Rachel Oldroyd, Managing Editor of The Bureau of Investigative Journalism; Peter Kruse, Founder of CSIS Security Group A/S; Ryan Gallagher, investigative Reporter of The Intercept; Capitán Alberto Redondo, Guardia Civil; Prof. Bart Preneel, KU Leuven; Raoul Chiesa, Security Brokers SCpA, CyberDefcon Ltd.; Prof.
    [Show full text]
  • Applying Library Values to Emerging Technology Decision-Making in the Age of Open Access, Maker Spaces, and the Ever-Changing Library
    ACRL Publications in Librarianship No. 72 Applying Library Values to Emerging Technology Decision-Making in the Age of Open Access, Maker Spaces, and the Ever-Changing Library Editors Peter D. Fernandez and Kelly Tilton Association of College and Research Libraries A division of the American Library Association Chicago, Illinois 2018 The paper used in this publication meets the minimum requirements of Ameri- can National Standard for Information Sciences–Permanence of Paper for Print- ed Library Materials, ANSI Z39.48-1992. ∞ Cataloging-in-Publication data is on file with the Library of Congress. Copyright ©2018 by the Association of College and Research Libraries. All rights reserved except those which may be granted by Sections 107 and 108 of the Copyright Revision Act of 1976. Printed in the United States of America. 22 21 20 19 18 5 4 3 2 1 Contents Contents Introduction .......................................................................................................ix Peter Fernandez, Head, LRE Liaison Programs, University of Tennessee Libraries Kelly Tilton, Information Literacy Instruction Librarian, University of Tennessee Libraries Part I Contemplating Library Values Chapter 1. ..........................................................................................................1 The New Technocracy: Positioning Librarianship’s Core Values in Relationship to Technology Is a Much Taller Order Than We Think John Buschman, Dean of University Libraries, Seton Hall University Chapter 2. ........................................................................................................27
    [Show full text]
  • Online-Quellen Nutzen: Recherche Im Internet
    ONLINE-QUELLEN NUTZEN: RECHERCHE IM INTERNET Katharina Gilarski, Verena Müller, Martin Nissen (Stand: 08/2020) Inhaltsverzeichnis 1 Recherche mit Universalsuchmaschinen ............................................................. 2 1.1 Clever googlen ......................................................................................................... 2 1.2 Suchoperatoren für die einfache Suche ................................................................... 2 1.3 Die erweiterte Suche mit Google .............................................................................. 3 1.4 Die umgekehrte Bildersuche mit Google .................................................................. 4 1.5 Das Geheimnis der Treffersortierung in Google ....................................................... 5 1.6 Alternative universelle Suchmaschinen .................................................................... 6 1.7 Surface vs. Deep Web ............................................................................................. 9 1.8 Vor- und Nachteile von universellen Suchmaschinen ............................................. 10 1.9 Bewertung von Internetquellen ............................................................................... 10 2 Recherche mit wissenschaftlichen Suchmaschinen ......................................... 11 2.1 Google Scholar: Googles Suchmaschine für Wissenschaftler ................................ 11 2.2 Suchmöglichkeiten von Google Scholar ................................................................
    [Show full text]
  • Fingerprinting Detection
    ACTIONS SPEAK LOUDER THAN WORDS: SEMI-SUPERVISED LEARNING FOR BROWSER FINGERPRINTING DETECTION Sarah Bird Vikas Mishra Steven Englehardt Mozilla INRIA Mozilla [email protected] [email protected] [email protected] Rob Willoughby David Zeber Walter Rudametkin University of British Colombia Mozilla INRIA [email protected] [email protected] [email protected] Martin Lopatka Mozilla [email protected] November 1, 2019 ABSTRACT As online tracking continues to grow, existing anti-tracking and fingerprinting detection techniques that require significant manual input must be augmented. Heuristic approaches to fingerprinting de- tection are precise but must be carefully curated. Supervised machine learning techniques proposed for detecting tracking require manually generated label-sets. Seeking to overcome these challenges, we present a semi-supervised machine learning approach for detecting fingerprinting scripts. Our approach is based on the core insight that fingerprinting scripts have similar patterns of API access when generating their fingerprints, even though their access patterns may not match exactly. Using this insight, we group scripts by their JavaScript (JS) execution traces and apply a semi-supervised approach to detect new fingerprinting scripts. We detail our methodology and demonstrate its abil- ity to identify the majority of scripts (>94.9%) identified by existing heuristic techniques. We also show that the approach expands beyond detecting known scripts by surfacing candidate scripts that arXiv:2003.04463v1 [cs.CR] 9 Mar 2020 are likely to include fingerprinting. Through an analysis of these candidate scripts we discovered fin- gerprinting scripts that were missed by heuristics and for which there are no heuristics. In particular, we identified over one hundred device-class fingerprinting scripts present on hundreds of domains.
    [Show full text]
  • Online Legal Research Handout
    The New On-Line Search Fundamentals 1. Search Process Natural language searches – Natural language searching is essentially the keyword searching we all know and love; the search results you get reflect algorithmic decision- making by the people who designed the search engine. Often the results you get will be based on the number of times your search terms appear in a document or on a web page. Boolean searches – Boolean searches are also referred to as “terms and connectors” searches. These are the searches using quotation marks, minus signs, slashes, and other symbols to “tell” the search engine exactly what you are searching for. Boolean searching gives you a way to limit your searches by dates, run searches in specific fields, and overall return more precise results. Natural Language v. Boolean searching compared - Most search engines (including Google, Bing, Westlaw Next, and Lexis Advance) are designed for natural language searching. - You can override natural language searching 1. If you know the terms and connectors used by that search engine. (I often click on “help” or something similar to see what terms and connectors the search engine uses.); or 2. If you can find the “advanced search” page for the search engine. - Natural language searching provides the “best” search results when You are not sure where to start and you need to figure out what search terms to use (i.e., you don’t know the words used on the webpages or in the documents you are looking for); You want a large number of search results you can sift through to figure out what kinds of results to expect; or You are researching a conceptual issue rather than a specific topic.
    [Show full text]
  • The Tor Dark Net
    PAPER SERIES: NO. 20 — SEPTEMBER 2015 The Tor Dark Net Gareth Owen and Nick Savage THE TOR DARK NET Gareth Owen and Nick Savage Copyright © 2015 by Gareth Owen and Nick Savage Published by the Centre for International Governance Innovation and the Royal Institute of International Affairs. The opinions expressed in this publication are those of the authors and do not necessarily reflect the views of the Centre for International Governance Innovation or its Board of Directors. This work is licensed under a Creative Commons Attribution — Non-commercial — No Derivatives License. To view this license, visit (www.creativecommons.org/licenses/by-nc- nd/3.0/). For re-use or distribution, please include this copyright notice. 67 Erb Street West 10 St James’s Square Waterloo, Ontario N2L 6C2 London, England SW1Y 4LE Canada United Kingdom tel +1 519 885 2444 fax +1 519 885 5450 tel +44 (0)20 7957 5700 fax +44 (0)20 7957 5710 www.cigionline.org www.chathamhouse.org TABLE OF CONTENTS vi About the Global Commission on Internet Governance vi About the Authors 1 Executive Summary 1 Introduction 2 Hidden Services 2 Related Work 3 Study of HSes 4 Content and Popularity Analysis 7 Deanonymization of Tor Users and HSes 8 Blocking of Tor 8 HS Blocking 9 Conclusion 9 Works Cited 12 About CIGI 12 About Chatham House 12 CIGI Masthead GLOBAL COMMISSION ON INTERNET GOVERNANCE PAPER SERIES: NO. 20 — SEPTEMBER 2015 ABOUT THE GLOBAL ABOUT THE AUTHORS COMMISSION ON INTERNET Gareth Owen is a senior lecturer in the School of GOVERNANCE Computing at the University of Portsmouth.
    [Show full text]
  • How Do Tor Users Interact with Onion Services?
    How Do Tor Users Interact With Onion Services? Philipp Winter Anne Edmundson Laura M. Roberts Princeton University Princeton University Princeton University Agnieszka Dutkowska-Zuk˙ Marshini Chetty Nick Feamster Independent Princeton University Princeton University Abstract messaging [4] and file sharing [15]. The Tor Project currently does not have data on the number of onion Onion services are anonymous network services that are service users, but Facebook reported in 2016 that more exposed over the Tor network. In contrast to conventional than one million users logged into its onion service in one Internet services, onion services are private, generally not month [20]. indexed by search engines, and use self-certifying domain Onion services differ from conventional web services names that are long and difficult for humans to read. In in four ways; First, they can only be accessed over the Tor this paper, we study how people perceive, understand, and network. Second, onion domains are hashes over their use onion services based on data from 17 semi-structured public key, which make them difficult to remember. Third, interviews and an online survey of 517 users. We find that the network path between client and the onion service is users have an incomplete mental model of onion services, typically longer, increasing latency and thus reducing the use these services for anonymity and have varying trust in performance of the service. Finally, onion services are onion services in general. Users also have difficulty dis- private by default, meaning that users must discover these covering and tracking onion sites and authenticating them. sites organically, rather than with a search engine.
    [Show full text]
  • Appendix I: Search Quality and Economies of Scale
    Appendix I: search quality and economies of scale Introduction 1. This appendix presents some detailed evidence concerning how general search engines compete on quality. It also examines network effects, scale economies and other barriers to search engines competing effectively on quality and other dimensions of competition. 2. This appendix draws on academic literature, submissions and internal documents from market participants. Comparing quality across search engines 3. As set out in Chapter 3, the main way that search engines compete for consumers is through various dimensions of quality, including relevance of results, privacy and trust, and social purpose or rewards.1 Of these dimensions, the relevance of search results is generally considered to be the most important. 4. In this section, we present consumer research and other evidence regarding how search engines compare on the quality and relevance of search results. Evidence on quality from consumer research 5. We reviewed a range of consumer research produced or commissioned by Google or Bing, to understand how consumers perceive the quality of these and other search engines. The following evidence concerns consumers’ overall quality judgements and preferences for particular search engines: 6. Google submitted the results of its Information Satisfaction tests. Google’s Information Satisfaction tests are quality comparisons that it carries out in the normal course of its business. In these tests, human raters score unbranded search results from mobile devices for a random set of queries from Google’s search logs. • Google submitted the results of 98 tests from the US between 2017 and 2020. Google outscored Bing in each test; the gap stood at 8.1 percentage points in the most recent test and 7.3 percentage points on average over the period.
    [Show full text]