DOCSLIB.ORG

Correcting for Selection Bias in Learning-To-Rank Systems

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Correcting for Selection Bias in Learning-To-Rank Systems Correcting for Selection Bias in Learning-to-rank Systems Zohreh Ovaisi Ragib Ahsan Yifan Zhang [email protected] [email protected] [email protected] University of Illinois at Chicago University of Illinois at Chicago Sun Yat-sen University Kathryn Vasilaky Elena Zheleva [email protected] [email protected] California Polytechnic State University of Illinois at Chicago University ABSTRACT the work on unbiasing the parameter estimation for learning-to- Click data collected by modern recommendation systems are an rank systems has focused on position bias [29], the bias caused by important source of observational data that can be utilized to train the position where a result was displayed to a user. Position bias learning-to-rank (LTR) systems. However, these data suffer from a makes higher ranked results (e.g., top search engine results) more number of biases that can result in poor performance for LTR sys- likely to be clicked even if they are not the most relevant. tems. Recent methods for bias correction in such systems mostly Algorithms that correct for position bias typically assume that focus on position bias, the fact that higher ranked results (e.g., top all relevant results have non-zero probability of being observed search engine results) are more likely to be clicked even if they are (and thus clicked) by the user and focus on boosting the relevance not the most relevant results given a user’s query. Less attention of lower ranked relevant results [29]. However, users rarely have has been paid to correcting for selection bias, which occurs because the chance to observe all relevant results, either because the sys- clicked documents are reflective of what documents have been tem chose toshow a truncated list of top k recommended results or shown to the user in the first place. Here, we propose new coun- because users do not spend the time to peruse through tens to hun- terfactual approaches which adapt Heckman’s two-stage method dreds of ranked results. In this case, lower ranked, relevant results and accounts for selection and position bias in LTR systems. Our have zero probability of being observed (and clicked) and never get empirical evaluation shows that our proposed methods are much the chance to be boosted in LTR systems. This leads to selection more robust to noise and have better accuracy compared to exist- bias in clicked results which is the focus of our work. ing unbiased LTR algorithms, especially when there is moderate Here, we frame the problem of learning to rank as a counterfac- to no position bias. tual problem of predicting whether a document would have been clicked had it been observed. In order to recover from selection bias KEYWORDS for clicked documents, we focus on identifying the relevant docu- ments that were never shown to users. Our formulation is different recommender systems, learning-to-rank, position bias, selection from previous counterfactual formulations which correct for posi- bias tion bias and study the likelihood of a document being clicked had it been placed in a higher position given that it was placed in a lower position [29]. 1 INTRODUCTION Here, we propose a general framework for recovering from se- The abundance of data found online has inspired new lines of in- lection bias that stems from both limited choices given to users and quiry about human behavior and the development of machine-learning position bias. First, we propose Heckmanrank, an algorithm for ad- algorithms that learn individual preferences from such data. Pat- dressing selection bias in the context of learning-to-rank systems. arXiv:2001.11358v2 [cs.IR] 12 May 2020 terns in such data are often driven by the underlying algorithms By adapting Heckman’s two-stage method, an econometric toolfor supporting online platforms, rather than naturally-occurring user addressing selection bias, we account for the limited choice given behavior. For example, interaction data from social media news to users and the fact that some items are more likely to be shownto feeds, such as user clicks and comments on posts, reflect not only a user than others. Because this correction method is very general, latent user interests but also news feed personalization and what it is applicable to any type of selection bias in which the system’s the underlying algorithms chose to show to users in the first place. decision to show documents can be learned from features. Because Such data in turn are used to train new news feed algorithms, prop- Heckmanrank treats selection as a binary variable, we propose two agating the bias further [9]. This can lead to phenomena such as bias-correcting ensembles that account for the nuanced probability filter bubbles and echo chambers and can challenge the validity of of being selected due to position bias and combine Heckmanrank social science research that relies on found data [26, 30]. with existing position-bias correction methods. One of the places where these biases surface is in personalized Our experimental evaluations demonstrate the utility of our pro- recommender systems whose goal is to learn user preferences from posed method when compared to state-of-the-art algorithms for available interaction data. These systems typically rely on learning unbiased learning-to-rank. Our ensemble methods have better ac- procedures to estimate the parameters of new ranking algorithms curacy compared to existing unbiased LTR algorithms under real- that are capable of ranking items based on inferred user prefer- istic selection bias assumptions, especially when the position bias ences, in a process known as learning-to-rank (LTR) [32]. Much of 1 Zohreh Ovaisi, Ragib Ahsan, Yifan Zhang, Kathryn Vasilaky, Elena Zheleva is not severe. Moreover, Heckmanrank is more robust to noise than bias, i.e., when a variable can affect both treatment and control. We both ensemble methods and position-bias correcting methods across leverage this work in our discussion of identifiability under selec- difference position bias assumptions. The experiments also show tion bias. that selection bias affects the performance of LTR systems even in The most related work to our context are studies by Hernández- the absence of position bias, and Heckmanrank is able to correct Lobato et al. [22], Schnabel et al. [37], Wang et al. [43]. Both Schn- for it. abel et al. [37] and Hernández-Lobato et al. [22] use a matrix factor- ization model to represent data (ratings by users) that are missing not-at-random, where Schnabel et al. [37] outperform Hernández- 2 RELATED WORK Lobato et al. [22]. More recently, Joachims et al. [29] propose a Here, we provide the context for our work and present the three position debiasing approach in the context of learning-to-rank sys- areas that best encompass our problem: bias in recommender sys- tems as a more general approach compared to Schnabel et al. [37]. tems, selection bias correction, and unbiased learning-to-rank. Throughout, we compare our results to Joachims et al. [29], al- Bias in recommender system. Many technological platforms, though, it should be noted that the latter deals with a more specific such as recommendation systems, tailor items to users by filter- bias - position bias - than what we address here. Finally, Wang et al. ing and ranking information according to user history. This pro- [43] address selection bias due to confounding, whereas we address cess influences the way users interact with the system and how selection bias that is treatment-dependent only. the data collected from users is fed back to the system and can Unbiased learning-to-rank. The problem we study here in- lead to several types of biases. Chaney et al. [9] explore a closely vestigates debiasing data in learning-to-rank systems. There are related problem called algorithmic confounding bias, where live two approaches to LTR systems, offline and online, and the work systems are retrained to incorporate data that was influenced by we propose here falls in the category of offline LTR systems. the recommendation algorithm itself. Their study highlights the Offline LTR systems learn a ranking model from historical click fact that training recommendation platforms with naive data that data and interpret clicks as absolute relevance indicators [2, 7, 12, are not debiased can cause a severe decrease in the utility of such 16, 24, 27–29, 36, 41, 42]. Offline approaches must contend with systems. For example, “echo chambers” are consequence of this the many biases that found data are subject to, including position problem [17, 20], where users are limited to an increasingly nar- and selection bias, among others. For example, Wang et al. [41] use rower choice set over time which can lead to a phenomenon called a propensity weighting approach to overcome position bias. Simi- polarization [18]. Popularity bias, is another bias affecting recom- larly, Joachims et al. [29] propose a method to correct for position mender system that is studied by Celma and Cano [8]. Popularity bias, by augmenting SV Mrank learning with an Inverse Propensity bias refers to the idea that a recommender system will display the Score defined for clicks rather than queries. They demonstrate that most popular items to a user, even if they are not the most rele- Propensity-Weighted SV Mrank outperforms a standard Ranking vant to a user’s query. Recommender systems can also affect users SV Mrank by accounting for position bias. More recently Agarwal decision making process, known as decision bias, and Chen et al. et al. [1] proposed nDCG SV Mrank that outperforms Propensity- [13] show how understanding this bias can improve recommender Weighted SV Mrank [29], but only when position bias is severe. We systems. Position bias is yet another type of bias that is studied in show that our proposed algorithm outperforms [29] when position the context of learning-to-rank systems and refers to documents bias is not severe.
Load more

Recommended publications
  • A Task-Based Taxonomy of Cognitive Biases for Information Visualization
    A Task-based Taxonomy of Cognitive Biases for Information Visualization Evanthia Dimara, Steven Franconeri, Catherine Plaisant, Anastasia Bezerianos, and Pierre Dragicevic Three kinds of limitations The Computer The Display 2 Three kinds of limitations The Computer The Display The Human 3 Three kinds of limitations: humans • Human vision ️ has limitations • Human reasoning 易 has limitations The Human 4 ️Perceptual bias Magnitude estimation 5 ️Perceptual bias Magnitude estimation Color perception 6 易 Cognitive bias Behaviors when humans consistently behave irrationally Pohl’s criteria distilled: • Are predictable and consistent • People are unaware they’re doing them • Are not misunderstandings 7 Ambiguity effect, Anchoring or focalism, Anthropocentric thinking, Anthropomorphism or personification, Attentional bias, Attribute substitution, Automation bias, Availability heuristic, Availability cascade, Backfire effect, Bandwagon effect, Base rate fallacy or Base rate neglect, Belief bias, Ben Franklin effect, Berkson's paradox, Bias blind spot, Choice-supportive bias, Clustering illusion, Compassion fade, Confirmation bias, Congruence bias, Conjunction fallacy, Conservatism (belief revision), Continued influence effect, Contrast effect, Courtesy bias, Curse of knowledge, Declinism, Decoy effect, Default effect, Denomination effect, Disposition effect, Distinction bias, Dread aversion, Dunning–Kruger effect, Duration neglect, Empathy gap, End-of-history illusion, Endowment effect, Exaggerated expectation, Experimenter's or expectation bias,
    [Show full text]
  • Cognitive Bias Mitigation: How to Make Decision-Making More Rational?
    Cognitive Bias Mitigation: How to make decision-making more rational? Abstract Cognitive biases distort judgement and adversely impact decision-making, which results in economic inefficiencies. Initial attempts to mitigate these biases met with little success. However, recent studies which used computer games and educational videos to train people to avoid biases (Clegg et al., 2014; Morewedge et al., 2015) showed that this form of training reduced selected cognitive biases by 30 %. In this work I report results of an experiment which investigated the debiasing effects of training on confirmation bias. The debiasing training took the form of a short video which contained information about confirmation bias, its impact on judgement, and mitigation strategies. The results show that participants exhibited confirmation bias both in the selection and processing of information, and that debiasing training effectively decreased the level of confirmation bias by 33 % at the 5% significance level. Key words: Behavioural economics, cognitive bias, confirmation bias, cognitive bias mitigation, confirmation bias mitigation, debiasing JEL classification: D03, D81, Y80 1 Introduction Empirical research has documented a panoply of cognitive biases which impair human judgement and make people depart systematically from models of rational behaviour (Gilovich et al., 2002; Kahneman, 2011; Kahneman & Tversky, 1979; Pohl, 2004). Besides distorted decision-making and judgement in the areas of medicine, law, and military (Nickerson, 1998), cognitive biases can also lead to economic inefficiencies. Slovic et al. (1977) point out how they distort insurance purchases, Hyman Minsky (1982) partly blames psychological factors for economic cycles. Shefrin (2010) argues that confirmation bias and some other cognitive biases were among the significant factors leading to the global financial crisis which broke out in 2008.
    [Show full text]
  • THE ROLE of PUBLICATION SELECTION BIAS in ESTIMATES of the VALUE of a STATISTICAL LIFE W
    THE ROLE OF PUBLICATION SELECTION BIAS IN ESTIMATES OF THE VALUE OF A STATISTICAL LIFE w. k i p vi s c u s i ABSTRACT Meta-regression estimates of the value of a statistical life (VSL) controlling for publication selection bias often yield bias-corrected estimates of VSL that are substantially below the mean VSL estimates. Labor market studies using the more recent Census of Fatal Occu- pational Injuries (CFOI) data are subject to less measurement error and also yield higher bias-corrected estimates than do studies based on earlier fatality rate measures. These re- sultsareborneoutbythefindingsforalargesampleofallVSLestimatesbasedonlabor market studies using CFOI data and for four meta-analysis data sets consisting of the au- thors’ best estimates of VSL. The confidence intervals of the publication bias-corrected estimates of VSL based on the CFOI data include the values that are currently used by government agencies, which are in line with the most precisely estimated values in the literature. KEYWORDS: value of a statistical life, VSL, meta-regression, publication selection bias, Census of Fatal Occupational Injuries, CFOI JEL CLASSIFICATION: I18, K32, J17, J31 1. Introduction The key parameter used in policy contexts to assess the benefits of policies that reduce mortality risks is the value of a statistical life (VSL).1 This measure of the risk-money trade-off for small risks of death serves as the basis for the standard approach used by government agencies to establish monetary benefit values for the predicted reductions in mortality risks from health, safety, and environmental policies. Recent government appli- cations of the VSL have used estimates in the $6 million to $10 million range, where these and all other dollar figures in this article are in 2013 dollars using the Consumer Price In- dex for all Urban Consumers (CPI-U).
    [Show full text]
  • Bias and Fairness in NLP
    Bias and Fairness in NLP Margaret Mitchell Kai-Wei Chang Vicente Ordóñez Román Google Brain UCLA University of Virginia Vinodkumar Prabhakaran Google Brain Tutorial Outline ● Part 1: Cognitive Biases / Data Biases / Bias laundering ● Part 2: Bias in NLP and Mitigation Approaches ● Part 3: Building Fair and Robust Representations for Vision and Language ● Part 4: Conclusion and Discussion “Bias Laundering” Cognitive Biases, Data Biases, and ML Vinodkumar Prabhakaran Margaret Mitchell Google Brain Google Brain Andrew Emily Simone Parker Lucy Ben Elena Deb Timnit Gebru Zaldivar Denton Wu Barnes Vasserman Hutchinson Spitzer Raji Adrian Brian Dirk Josh Alex Blake Hee Jung Hartwig Blaise Benton Zhang Hovy Lovejoy Beutel Lemoine Ryu Adam Agüera y Arcas What’s in this tutorial ● Motivation for Fairness research in NLP ● How and why NLP models may be unfair ● Various types of NLP fairness issues and mitigation approaches ● What can/should we do? What’s NOT in this tutorial ● Definitive answers to fairness/ethical questions ● Prescriptive solutions to fix ML/NLP (un)fairness What do you see? What do you see? ● Bananas What do you see? ● Bananas ● Stickers What do you see? ● Bananas ● Stickers ● Dole Bananas What do you see? ● Bananas ● Stickers ● Dole Bananas ● Bananas at a store What do you see? ● Bananas ● Stickers ● Dole Bananas ● Bananas at a store ● Bananas on shelves What do you see? ● Bananas ● Stickers ● Dole Bananas ● Bananas at a store ● Bananas on shelves ● Bunches of bananas What do you see? ● Bananas ● Stickers ● Dole Bananas ● Bananas
    [Show full text]
  • Why So Confident? the Influence of Outcome Desirability on Selective Exposure and Likelihood Judgment
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by The University of North Carolina at Greensboro Archived version from NCDOCKS Institutional Repository http://libres.uncg.edu/ir/asu/ Why so confident? The influence of outcome desirability on selective exposure and likelihood judgment Authors Paul D. Windschitl , Aaron M. Scherer a, Andrew R. Smith , Jason P. Rose Abstract Previous studies that have directly manipulated outcome desirability have often found little effect on likelihood judgments (i.e., no desirability bias or wishful thinking). The present studies tested whether selections of new information about outcomes would be impacted by outcome desirability, thereby biasing likelihood judgments. In Study 1, participants made predictions about novel outcomes and then selected additional information to read from a buffet. They favored information supporting their prediction, and this fueled an increase in confidence. Studies 2 and 3 directly manipulated outcome desirability through monetary means. If a target outcome (randomly preselected) was made especially desirable, then participants tended to select information that supported the outcome. If made undesirable, less supporting information was selected. Selection bias was again linked to subsequent likelihood judgments. These results constitute novel evidence for the role of selective exposure in cases of overconfidence and desirability bias in likelihood judgments. Paul D. Windschitl , Aaron M. Scherer a, Andrew R. Smith , Jason P. Rose (2013) "Why so confident? The influence of outcome desirability on selective exposure and likelihood judgment" Organizational Behavior and Human Decision Processes 120 (2013) 73–86 (ISSN: 0749-5978) Version of Record available @ DOI: (http://dx.doi.org/10.1016/j.obhdp.2012.10.002) Why so confident? The influence of outcome desirability on selective exposure and likelihood judgment q a,⇑ a c b Paul D.
    [Show full text]
  • Correcting Sampling Bias in Non-Market Valuation with Kernel Mean Matching
    CORRECTING SAMPLING BIAS IN NON-MARKET VALUATION WITH KERNEL MEAN MATCHING Rui Zhang Department of Agricultural and Applied Economics University of Georgia [email protected] Selected Paper prepared for presentation at the 2017 Agricultural & Applied Economics Association Annual Meeting, Chicago, Illinois, July 30 - August 1 Copyright 2017 by Rui Zhang. All rights reserved. Readers may make verbatim copies of this document for non-commercial purposes by any means, provided that this copyright notice appears on all such copies. Abstract Non-response is common in surveys used in non-market valuation studies and can bias the parameter estimates and mean willingness to pay (WTP) estimates. One approach to correct this bias is to reweight the sample so that the distribution of the characteristic variables of the sample can match that of the population. We use a machine learning algorism Kernel Mean Matching (KMM) to produce resampling weights in a non-parametric manner. We test KMM’s performance through Monte Carlo simulations under multiple scenarios and show that KMM can effectively correct mean WTP estimates, especially when the sample size is small and sampling process depends on covariates. We also confirm KMM’s robustness to skewed bid design and model misspecification. Key Words: contingent valuation, Kernel Mean Matching, non-response, bias correction, willingness to pay 2 1. Introduction Nonrandom sampling can bias the contingent valuation estimates in two ways. Firstly, when the sample selection process depends on the covariate, the WTP estimates are biased due to the divergence between the covariate distributions of the sample and the population, even the parameter estimates are consistent; this is usually called non-response bias.
    [Show full text]
  • Evaluation of Selection Bias in an Internet-Based Study of Pregnancy Planners
    HHS Public Access Author manuscript Author ManuscriptAuthor Manuscript Author Epidemiology Manuscript Author . Author manuscript; Manuscript Author available in PMC 2016 April 04. Published in final edited form as: Epidemiology. 2016 January ; 27(1): 98–104. doi:10.1097/EDE.0000000000000400. Evaluation of Selection Bias in an Internet-based Study of Pregnancy Planners Elizabeth E. Hatcha, Kristen A. Hahna, Lauren A. Wisea, Ellen M. Mikkelsenb, Ramya Kumara, Matthew P. Foxa, Daniel R. Brooksa, Anders H. Riisb, Henrik Toft Sorensenb, and Kenneth J. Rothmana,c aDepartment of Epidemiology, Boston University School of Public Health, Boston, MA bDepartment of Clinical Epidemiology, Aarhus University Hospital, Aarhus, Denmark cRTI Health Solutions, Durham, NC Abstract Selection bias is a potential concern in all epidemiologic studies, but it is usually difficult to assess. Recently, concerns have been raised that internet-based prospective cohort studies may be particularly prone to selection bias. Although use of the internet is efficient and facilitates recruitment of subjects that are otherwise difficult to enroll, any compromise in internal validity would be of great concern. Few studies have evaluated selection bias in internet-based prospective cohort studies. Using data from the Danish Medical Birth Registry from 2008 to 2012, we compared six well-known perinatal associations (e.g., smoking and birth weight) in an inter-net- based preconception cohort (Snart Gravid n = 4,801) with the total population of singleton live births in the registry (n = 239,791). We used log-binomial models to estimate risk ratios (RRs) and 95% confidence intervals (CIs) for each association. We found that most results in both populations were very similar.
    [Show full text]
  • Testing for Selection Bias IZA DP No
    IZA DP No. 8455 Testing for Selection Bias Joonhwi Joo Robert LaLonde September 2014 DISCUSSION PAPER SERIES Forschungsinstitut zur Zukunft der Arbeit Institute for the Study of Labor Testing for Selection Bias Joonhwi Joo University of Chicago Robert LaLonde University of Chicago and IZA Discussion Paper No. 8455 September 2014 IZA P.O. Box 7240 53072 Bonn Germany Phone: +49-228-3894-0 Fax: +49-228-3894-180 E-mail: [email protected] Any opinions expressed here are those of the author(s) and not those of IZA. Research published in this series may include views on policy, but the institute itself takes no institutional policy positions. The IZA research network is committed to the IZA Guiding Principles of Research Integrity. The Institute for the Study of Labor (IZA) in Bonn is a local and virtual international research center and a place of communication between science, politics and business. IZA is an independent nonprofit organization supported by Deutsche Post Foundation. The center is associated with the University of Bonn and offers a stimulating research environment through its international network, workshops and conferences, data service, project support, research visits and doctoral program. IZA engages in (i) original and internationally competitive research in all fields of labor economics, (ii) development of policy concepts, and (iii) dissemination of research results and concepts to the interested public. IZA Discussion Papers often represent preliminary work and are circulated to encourage discussion. Citation of such a paper should account for its provisional character. A revised version may be available directly from the author. IZA Discussion Paper No.
    [Show full text]
  • Correcting Sample Selection Bias by Unlabeled Data
    Correcting Sample Selection Bias by Unlabeled Data Jiayuan Huang Alexander J. Smola Arthur Gretton School of Computer Science NICTA, ANU MPI for Biological Cybernetics Univ. of Waterloo, Canada Canberra, Australia T¨ubingen, Germany [email protected] [email protected] [email protected] Karsten M. Borgwardt Bernhard Scholkopf¨ Ludwig-Maximilians-University MPI for Biological Cybernetics Munich, Germany T¨ubingen, Germany [email protected]fi.lmu.de [email protected] Abstract We consider the scenario where training and test data are drawn from different distributions, commonly referred to as sample selection bias. Most algorithms for this setting try to first recover sampling distributions and then make appro- priate corrections based on the distribution estimate. We present a nonparametric method which directly produces resampling weights without distribution estima- tion. Our method works by matching distributions between training and testing sets in feature space. Experimental results demonstrate that our method works well in practice. 1 Introduction The default assumption in many learning scenarios is that training and test data are independently and identically (iid) drawn from the same distribution. When the distributions on training and test set do not match, we are facing sample selection bias or covariate shift. Specifically, given a domain of patterns X and labels Y, we obtain training samples Z = (x , y ),..., (x , y ) X Y from { 1 1 m m } ⊆ × a Borel probability distribution Pr(x, y), and test samples Z′ = (x′ , y′ ),..., (x′ ′ , y′ ′ ) X Y { 1 1 m m } ⊆ × drawn from another such distribution Pr′(x, y). Although there exists previous work addressing this problem [2, 5, 8, 9, 12, 16, 20], sample selection bias is typically ignored in standard estimation algorithms.
    [Show full text]
  • The Bias Bias in Behavioral Economics
    Review of Behavioral Economics, 2018, 5: 303–336 The Bias Bias in Behavioral Economics Gerd Gigerenzer∗ Max Planck Institute for Human Development, Lentzeallee 94, 14195 Berlin, Germany; [email protected] ABSTRACT Behavioral economics began with the intention of eliminating the psychological blind spot in rational choice theory and ended up portraying psychology as the study of irrationality. In its portrayal, people have systematic cognitive biases that are not only as persis- tent as visual illusions but also costly in real life—meaning that governmental paternalism is called upon to steer people with the help of “nudges.” These biases have since attained the status of truisms. In contrast, I show that such a view of human nature is tainted by a “bias bias,” the tendency to spot biases even when there are none. This may occur by failing to notice when small sample statistics differ from large sample statistics, mistaking people’s ran- dom error for systematic error, or confusing intelligent inferences with logical errors. Unknown to most economists, much of psycho- logical research reveals a different portrayal, where people appear to have largely fine-tuned intuitions about chance, frequency, and framing. A systematic review of the literature shows little evidence that the alleged biases are potentially costly in terms of less health, wealth, or happiness. Getting rid of the bias bias is a precondition for psychology to play a positive role in economics. Keywords: Behavioral economics, Biases, Bounded Rationality, Imperfect information Behavioral economics began with the intention of eliminating the psycho- logical blind spot in rational choice theory and ended up portraying psychology as the source of irrationality.
    [Show full text]
  • Do Visitors of Product Evaluation Portals Select Reviews in a Biased Manner? Cyberpsychology: Journal of Psychosocial Research on Cyberspace, 15(1), Article 4
    Winter, K., Zapf, B., Hütter, M., Tichy, N., & Sassenberg, K. (2021). Selective exposure in action: Do visitors of product evaluation portals select reviews in a biased manner? Cyberpsychology: Journal of Psychosocial Research on Cyberspace, 15(1), article 4. https://doi.org/10.5817/CP2021-1-4 Selective Exposure in Action: Do Visitors of Product Evaluation Portals Select Reviews in a Biased Manner? Kevin Winter1, Birka Zapf1,2, Mandy Hütter2, Nicolas Tichy3, & Kai Sassenberg1,2 1 Leibniz-Institut für Wissensmedien, Tübingen, Germany 2 University of Tübingen, Tübingen, Germany 3 Ludwig Maximilian University of Munich, Munich, Germany Abstract Most people in industrialized countries regularly purchase products online. Consumers often rely on previous customers’ reviews to make purchasing decisions. The current research investigates whether potential online customers select these reviews in a biased way and whether typical interface properties of product evaluation portals foster biased selection. Based on selective exposure research, potential online customers should have a bias towards selecting positive reviews when they have an initial preference for a product. We tested this prediction across five studies (total N = 1376) while manipulating several typical properties of the review selection interface that should – according to earlier findings – facilitate biased selection. Across all studies, we found some evidence for a bias in favor of selecting positive reviews, but the aggregated effect was non-significant in an internal meta- analysis. Contrary to our hypothesis and not replicating previous research, none of the interface properties that were assumed to increase biased selection led to the predicted effects. Overall, the current research suggests that biased information selection, which has regularly been found in many other contexts, only plays a minor role in online review selection.
    [Show full text]
  • Bias and Confounding • Describe the Key Types of Bias
    M1 - PopMed OBJECTIVES Bias and Confounding • Describe the key types of bias • Identify sources of bias in study design, data collection and analysis Saba Masho, MD, MPH, DrPH Department of Epidemiology and Community Health • Identify confounders and methods for [email protected] controlling confounders 1 2 Bias Types: • Bias is any systematic error in an epidemiologic study that results in an 1. Selection incorrect estimate of association between risk factor and outcome. 2. Information/measurement • It is introduced in the design of the study including, data collection, analysis, 3. Confounding interpretation, publication or review of data. • It is impossible to correct for bias during the analysis and may also be hard to evaluate. Thus, studies need to be carefully designed. 3 4 SELECTION BIAS SELECTION BIAS • A systematic error that arises in the process of selecting the study populations • May also occur when membership in one group differs systematically from the general population or • May occur when different criteria is used to select control group, called membership bias cases/controls or exposed/non exposed – E.g. selecting cases from inner hospitals and controls from – E.g. A study in uterine cancer excluding patients with sub-urban hospitals hysterectomy from cases but not from control • The differences in hospital admission between cases • Detection bias: occurs when the risk factor under and controls may conceal an association in a study, investigation requires thorough diagnostic this is called “Berkson’s bias” or “admission
    [Show full text]