DOCSLIB.ORG

Bias and Confounding • Describe the Key Types of Bias

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Bias and Confounding • Describe the Key Types of Bias M1 - PopMed OBJECTIVES Bias and Confounding • Describe the key types of bias • Identify sources of bias in study design, data collection and analysis Saba Masho, MD, MPH, DrPH Department of Epidemiology and Community Health • Identify confounders and methods for [email protected] controlling confounders 1 2 Bias Types: • Bias is any systematic error in an epidemiologic study that results in an 1. Selection incorrect estimate of association between risk factor and outcome. 2. Information/measurement • It is introduced in the design of the study including, data collection, analysis, 3. Confounding interpretation, publication or review of data. • It is impossible to correct for bias during the analysis and may also be hard to evaluate. Thus, studies need to be carefully designed. 3 4 SELECTION BIAS SELECTION BIAS • A systematic error that arises in the process of selecting the study populations • May also occur when membership in one group differs systematically from the general population or • May occur when different criteria is used to select control group, called membership bias cases/controls or exposed/non exposed – E.g. selecting cases from inner hospitals and controls from – E.g. A study in uterine cancer excluding patients with sub-urban hospitals hysterectomy from cases but not from control • The differences in hospital admission between cases • Detection bias: occurs when the risk factor under and controls may conceal an association in a study, investigation requires thorough diagnostic this is called “Berkson’s bias” or “admission rate assessment. bias” – E.g. Benign breast tumor usually undergo detailed diagnostic assessment. Women with benign lesion are more likely to be diagnosed with the illness … due to the assessment 5 6 SELECTION BIAS SELECTION BIAS • Migration bias: study subjects in one subgroup of a • Sampling bias: when the selected sample for the cohort leave their original group, dropping out of the study is systematically different from the population study or moving to one of the other groups they are designed to represent – Loss to follow-up bias – most common in cohort and experimental studies – patients are lost from follow up • Lead –time bias: lead-time is a period between the detection of medical condition by screening and – Crossover – move of patients from one group to when the condition is diagnosed when it is another in the cohort during their follow-up symptomatic. Screening detects the problem before it • Susceptibility bias or assembly bias: groups being is symptomatic therefore, - better survival than those compared are not equally susceptible to the outcome entering when diagnosed of interest – E.g. Cases are significantly older than controls in cancer research – older people are more likely to have cancer 7 8 INFORMATION BIAS SELECTION BIAS • Also called measurement or observation bias • Length time bias: occurs when the disease of interest has slow progression. If such types of conditions are found through screening versus when • Systematic differences in the way exposure or symptomatic their prognosis is different affecting outcome data are obtained. mortality. • Measurement bias: methods of measurement are dissimilar in different groups of patients. • Compliance bias: common in experimental studies – one group is less compliant than the other • Recall bias: difference in remembering systematically affecting the result of the study. and reporting • Interviewer bias: systematic difference in soliciting, 9 recording, or interpreting of information 10 INFORMATION BIAS CONTROL OF BIAS • Misclassification: occurs when subjects are 1) Study design mistakenly categorized by exposure or disease • Choice of the study population: e.g. hospital controls status. for hospital based studies • E.g. smoking in pregnancy – because patients deny smoking – they may be misclassified as non smokers. • Choosing well defined and stable population. – Random (non-differential): when the factor(s) • Choosing population that might be interested in the are equally distributed in the study groups – no study. effect • Clear and uniform criteria for selection – Non-random (differential): unequally distributed between the study groups – under or over- estimate 11 12 CONTROL OF BIAS CONTROL OF BIAS 2) Methods of data collection • Include several items directed to the same question • Data collection instruments: leave less on the questionnaire. room for interpretation by the subjects-providing clear • Record time to find out if interviewers are spending options. Select best method for data collection – e.g. more time on some subjects over the others. high cholesterol – use lab data instead of interview. • Reliability scores – subjects with unreliable • Blinding - chart abstractors, interviewers or subjects information may be excluded. • Provide standardized training and 3) Sources of exposure and disease information written protocol for data collection. • Use of pre-existing record is the most unbiased because it was recorded before the onset of the study. However, records may not have complete information on all factors. • Having multiple sources of data to verify accuracy is 13 14 beneficial. Remarks in Evaluation of the Role of Bias Confounding • To maximum possible, potential sources of bias • Confounders are factors that are associated both to should be eliminated through rigorous design and the risk factor and outcome of interest. These factors conduct of the study can be independently risk factors not an intermediate • Existing biases and potential effects of the biases link. has to be discussed in the report • Investigators may want to build strategies in the methodologies to assess the extent of the effect of the potential bias • In evaluating the role of bias, the type of study, its design and nature of the results has to be examined 15 16 Example: sex is a confounder in the association between Confounding smoking and breast cancer. • Can lead to an over or underestimate OR = 54*148/62*156 = 0.826 • Existence of confounders can be determined by comparing the crude estimate of association with controlled estimate – • The effect of confounding has to be considered at the design of the study by collecting the necessary information based on existing knowledge and utilizing previous studies OR = 1.55 for males OR = 1.33 for females 17 18 Example: alcohol consumption and myocardial infarction (MI). • Mantel-Haenszel overall odds ratio (to adjust for •OR = 2.26 confounding): MH-OR (overall) • = [(4*140)/210] +[(50*8/210)] / [(60*6/210) + (2*150/210) ]= 1.454 OR = 1.00 for nonsmokers OR = 1.00 for smokers 19 20 Method of Controlling Confounders Randomization Can be done in the: • In large enough sample size all potential confounding known and unknown are evenly 1. Study design distributed between the study groups. 1. Randomization, 2. Restriction and • Is unique because of its ability to control for 3. Matching the effect of unknown confounders 2. Analysis • Imbalance because of small sample size or 1. Stratification chance can be controlled during analysis 2. Multivariate analysis 21 22 Matching Restriction • Restrict the study to a specified category or • Done in the design and analysis phase categories. – E.g. To control for the effect of age and sex in lung – E.g. If gender is a confounding variable, the study cancer and smoking, for each case of lung cancer can be restricted to one gender. with a particular age and sex, one or more controls • Creates a homogenous study population will be selected of the same age and gender • Is easy, convenient and inexpensive method • Matching is mostly used in case control studies Limitations of restriction include: – Reduction in the number of subjects • Maximize the sample size in respect to the potential confounder – Residual confounding – if the criteria is not strict – Limits the evaluation to the group - affecting generalizability 23 24 Matching as a Means of Controlling for Confounding Odds Ratio = b/c • Matching variables should meet the following criteria: – Related both to exposure and disease, so that they are potential confounders – Fairly strongly related to disease so that matching will have the potential for increasing the precision of estimation. – Readily available at little or no cost to the investigator • Common matching variables • gender • age group •race 25 26 Matching Matching Limitations: • Considering the benefit and limitation, matching – Difficult, expensive, and time-consuming to find should only be considered after careful analysis of its comparison with the set of characteristics limitations and benefits – Inability of evaluating factors that are used for • A matched data will lead to similarity between cases matching and controls with respect to the exposure. If this is not appropriately analyzed, it will provide an underestimate in the true association. Thus, – Does not control for other factors other than the matching in the design by itself will not control for the matching factors effect. – Reduces sample size for analysis because of • Therefore, matching should be accompanied by discordant pairs stratification or multivariate analysis. 27 28 Example: Example: • In a case-control study of menopausal estrogen use and • There were 39 pairs in which both the case and her endometrial cancer, 317 cases of endometrial cancer matching control used estrogens, and 150 pairs in which and controls (with other forms of cancers) were matched neither the case nor her matching control used on the basis of age at diagnosis (within 2 years) and estrogens.
Load more

Recommended publications
  • A Task-Based Taxonomy of Cognitive Biases for Information Visualization
    A Task-based Taxonomy of Cognitive Biases for Information Visualization Evanthia Dimara, Steven Franconeri, Catherine Plaisant, Anastasia Bezerianos, and Pierre Dragicevic Three kinds of limitations The Computer The Display 2 Three kinds of limitations The Computer The Display The Human 3 Three kinds of limitations: humans • Human vision ️ has limitations • Human reasoning 易 has limitations The Human 4 ️Perceptual bias Magnitude estimation 5 ️Perceptual bias Magnitude estimation Color perception 6 易 Cognitive bias Behaviors when humans consistently behave irrationally Pohl’s criteria distilled: • Are predictable and consistent • People are unaware they’re doing them • Are not misunderstandings 7 Ambiguity effect, Anchoring or focalism, Anthropocentric thinking, Anthropomorphism or personification, Attentional bias, Attribute substitution, Automation bias, Availability heuristic, Availability cascade, Backfire effect, Bandwagon effect, Base rate fallacy or Base rate neglect, Belief bias, Ben Franklin effect, Berkson's paradox, Bias blind spot, Choice-supportive bias, Clustering illusion, Compassion fade, Confirmation bias, Congruence bias, Conjunction fallacy, Conservatism (belief revision), Continued influence effect, Contrast effect, Courtesy bias, Curse of knowledge, Declinism, Decoy effect, Default effect, Denomination effect, Disposition effect, Distinction bias, Dread aversion, Dunning–Kruger effect, Duration neglect, Empathy gap, End-of-history illusion, Endowment effect, Exaggerated expectation, Experimenter's or expectation bias,
    [Show full text]
  • Cognitive Bias Mitigation: How to Make Decision-Making More Rational?
    Cognitive Bias Mitigation: How to make decision-making more rational? Abstract Cognitive biases distort judgement and adversely impact decision-making, which results in economic inefficiencies. Initial attempts to mitigate these biases met with little success. However, recent studies which used computer games and educational videos to train people to avoid biases (Clegg et al., 2014; Morewedge et al., 2015) showed that this form of training reduced selected cognitive biases by 30 %. In this work I report results of an experiment which investigated the debiasing effects of training on confirmation bias. The debiasing training took the form of a short video which contained information about confirmation bias, its impact on judgement, and mitigation strategies. The results show that participants exhibited confirmation bias both in the selection and processing of information, and that debiasing training effectively decreased the level of confirmation bias by 33 % at the 5% significance level. Key words: Behavioural economics, cognitive bias, confirmation bias, cognitive bias mitigation, confirmation bias mitigation, debiasing JEL classification: D03, D81, Y80 1 Introduction Empirical research has documented a panoply of cognitive biases which impair human judgement and make people depart systematically from models of rational behaviour (Gilovich et al., 2002; Kahneman, 2011; Kahneman & Tversky, 1979; Pohl, 2004). Besides distorted decision-making and judgement in the areas of medicine, law, and military (Nickerson, 1998), cognitive biases can also lead to economic inefficiencies. Slovic et al. (1977) point out how they distort insurance purchases, Hyman Minsky (1982) partly blames psychological factors for economic cycles. Shefrin (2010) argues that confirmation bias and some other cognitive biases were among the significant factors leading to the global financial crisis which broke out in 2008.
    [Show full text]
  • Bbm:978-1-59745-385-1/1.Pdf
    INDEX A information bias .........................................................29 loss to follow-up bias ....................................... 175, 181 Accuracy minimizing methods of ................................... 181–182 clinical research data ................................................309 nonresponse bias ..............................................175, 181 diagnostic tests .................................................132–133 overmatching bias ....................................................176 Additive models, interaction in. See Interaction recall/memory bias ...........................................176–177 Address transparency .....................................................256 sampling bias ......................................................... 6, 29 ADL and Barthel index .................................................156 types of Alignment of research questions .................................... 300 confounding bias ................................................180 Allele-sharing method ...........................................191–194 information bias .........................................176–179 Allocation concealment ............................................... 6, 98 intervention (exposure) bias ................173, 179–180 Allocative efficiency ........................................237–239, 241 selection bias ...................................24, 25, 173–176 Alpha spending.............................................................. 117 Biomarkers Analyses confounding ....................................................
    [Show full text]
  • THE ROLE of PUBLICATION SELECTION BIAS in ESTIMATES of the VALUE of a STATISTICAL LIFE W
    THE ROLE OF PUBLICATION SELECTION BIAS IN ESTIMATES OF THE VALUE OF A STATISTICAL LIFE w. k i p vi s c u s i ABSTRACT Meta-regression estimates of the value of a statistical life (VSL) controlling for publication selection bias often yield bias-corrected estimates of VSL that are substantially below the mean VSL estimates. Labor market studies using the more recent Census of Fatal Occu- pational Injuries (CFOI) data are subject to less measurement error and also yield higher bias-corrected estimates than do studies based on earlier fatality rate measures. These re- sultsareborneoutbythefindingsforalargesampleofallVSLestimatesbasedonlabor market studies using CFOI data and for four meta-analysis data sets consisting of the au- thors’ best estimates of VSL. The confidence intervals of the publication bias-corrected estimates of VSL based on the CFOI data include the values that are currently used by government agencies, which are in line with the most precisely estimated values in the literature. KEYWORDS: value of a statistical life, VSL, meta-regression, publication selection bias, Census of Fatal Occupational Injuries, CFOI JEL CLASSIFICATION: I18, K32, J17, J31 1. Introduction The key parameter used in policy contexts to assess the benefits of policies that reduce mortality risks is the value of a statistical life (VSL).1 This measure of the risk-money trade-off for small risks of death serves as the basis for the standard approach used by government agencies to establish monetary benefit values for the predicted reductions in mortality risks from health, safety, and environmental policies. Recent government appli- cations of the VSL have used estimates in the $6 million to $10 million range, where these and all other dollar figures in this article are in 2013 dollars using the Consumer Price In- dex for all Urban Consumers (CPI-U).
    [Show full text]
  • Bias and Fairness in NLP
    Bias and Fairness in NLP Margaret Mitchell Kai-Wei Chang Vicente Ordóñez Román Google Brain UCLA University of Virginia Vinodkumar Prabhakaran Google Brain Tutorial Outline ● Part 1: Cognitive Biases / Data Biases / Bias laundering ● Part 2: Bias in NLP and Mitigation Approaches ● Part 3: Building Fair and Robust Representations for Vision and Language ● Part 4: Conclusion and Discussion “Bias Laundering” Cognitive Biases, Data Biases, and ML Vinodkumar Prabhakaran Margaret Mitchell Google Brain Google Brain Andrew Emily Simone Parker Lucy Ben Elena Deb Timnit Gebru Zaldivar Denton Wu Barnes Vasserman Hutchinson Spitzer Raji Adrian Brian Dirk Josh Alex Blake Hee Jung Hartwig Blaise Benton Zhang Hovy Lovejoy Beutel Lemoine Ryu Adam Agüera y Arcas What’s in this tutorial ● Motivation for Fairness research in NLP ● How and why NLP models may be unfair ● Various types of NLP fairness issues and mitigation approaches ● What can/should we do? What’s NOT in this tutorial ● Definitive answers to fairness/ethical questions ● Prescriptive solutions to fix ML/NLP (un)fairness What do you see? What do you see? ● Bananas What do you see? ● Bananas ● Stickers What do you see? ● Bananas ● Stickers ● Dole Bananas What do you see? ● Bananas ● Stickers ● Dole Bananas ● Bananas at a store What do you see? ● Bananas ● Stickers ● Dole Bananas ● Bananas at a store ● Bananas on shelves What do you see? ● Bananas ● Stickers ● Dole Bananas ● Bananas at a store ● Bananas on shelves ● Bunches of bananas What do you see? ● Bananas ● Stickers ● Dole Bananas ● Bananas
    [Show full text]
  • Working Memory, Cognitive Miserliness and Logic As Predictors of Performance on the Cognitive Reflection Test
    Working Memory, Cognitive Miserliness and Logic as Predictors of Performance on the Cognitive Reflection Test Edward J. N. Stupple ([email protected]) Centre for Psychological Research, University of Derby Kedleston Road, Derby. DE22 1GB Maggie Gale ([email protected]) Centre for Psychological Research, University of Derby Kedleston Road, Derby. DE22 1GB Christopher R. Richmond ([email protected]) Centre for Psychological Research, University of Derby Kedleston Road, Derby. DE22 1GB Abstract Most participants respond that the answer is 10 cents; however, a slower and more analytic approach to the The Cognitive Reflection Test (CRT) was devised to measure problem reveals the correct answer to be 5 cents. the inhibition of heuristic responses to favour analytic ones. The CRT has been a spectacular success, attracting more Toplak, West and Stanovich (2011) demonstrated that the than 100 citations in 2012 alone (Scopus). This may be in CRT was a powerful predictor of heuristics and biases task part due to the ease of administration; with only three items performance - proposing it as a metric of the cognitive miserliness central to dual process theories of thinking. This and no requirement for expensive equipment, the practical thesis was examined using reasoning response-times, advantages are considerable. There have, moreover, been normative responses from two reasoning tasks and working numerous correlates of the CRT demonstrated, from a wide memory capacity (WMC) to predict individual differences in range of tasks in the heuristics and biases literature (Toplak performance on the CRT. These data offered limited support et al., 2011) to risk aversion and SAT scores (Frederick, for the view of miserliness as the primary factor in the CRT.
    [Show full text]
  • Why So Confident? the Influence of Outcome Desirability on Selective Exposure and Likelihood Judgment
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by The University of North Carolina at Greensboro Archived version from NCDOCKS Institutional Repository http://libres.uncg.edu/ir/asu/ Why so confident? The influence of outcome desirability on selective exposure and likelihood judgment Authors Paul D. Windschitl , Aaron M. Scherer a, Andrew R. Smith , Jason P. Rose Abstract Previous studies that have directly manipulated outcome desirability have often found little effect on likelihood judgments (i.e., no desirability bias or wishful thinking). The present studies tested whether selections of new information about outcomes would be impacted by outcome desirability, thereby biasing likelihood judgments. In Study 1, participants made predictions about novel outcomes and then selected additional information to read from a buffet. They favored information supporting their prediction, and this fueled an increase in confidence. Studies 2 and 3 directly manipulated outcome desirability through monetary means. If a target outcome (randomly preselected) was made especially desirable, then participants tended to select information that supported the outcome. If made undesirable, less supporting information was selected. Selection bias was again linked to subsequent likelihood judgments. These results constitute novel evidence for the role of selective exposure in cases of overconfidence and desirability bias in likelihood judgments. Paul D. Windschitl , Aaron M. Scherer a, Andrew R. Smith , Jason P. Rose (2013) "Why so confident? The influence of outcome desirability on selective exposure and likelihood judgment" Organizational Behavior and Human Decision Processes 120 (2013) 73–86 (ISSN: 0749-5978) Version of Record available @ DOI: (http://dx.doi.org/10.1016/j.obhdp.2012.10.002) Why so confident? The influence of outcome desirability on selective exposure and likelihood judgment q a,⇑ a c b Paul D.
    [Show full text]
  • 3. Studies of Colorectal Cancer Screening
    IARC HANDBOOKS COLORECTAL CANCER SCREENING VOLUME 17 This publication represents the views and expert opinions of an IARC Working Group on the Evaluation of Cancer-Preventive Strategies, which met in Lyon, 14–21 November 2017 LYON, FRANCE - 2019 IARC HANDBOOKS OF CANCER PREVENTION 3. STUDIES OF COLORECTAL CANCER SCREENING 3.1 Methodological considerations end-point of the RCT can be the incidence of the cancer of interest. Methods for colorectal cancer (CRC) screen- The observed effect of screening in RCTs is ing include endoscopic methods and stool-based dependent on, among other things, the partic- tests for blood. The two primary end-points for ipation in the intervention group and the limi- endoscopic CRC screening are (i) finding cancer tation of contamination of the control group. at an early stage (secondary prevention) and Low participation biases the estimate of effect (ii) finding and removing precancerous lesions towards its no-effect value, and therefore it must (adenomatous polyps), to reduce the incidence be evaluated and reported. Screening of controls of CRC (primary prevention). The primary by services outside of the RCT also dilutes the end-point for stool-based tests is finding cancer effect of screening on CRC incidence and/or at an early stage. Stool-based tests also have some mortality. If the screening modality being eval- ability to detect adenomatous polyps; therefore, a uated is widely used in clinical practice in secondary end-point of these tests is reducing the the region or regions where the RCT is being incidence of CRC. conducted, then contamination may be consid- erable, although it may be difficult and/or costly 3.1.1 Randomized controlled trials of to estimate its extent.
    [Show full text]
  • Correcting Sampling Bias in Non-Market Valuation with Kernel Mean Matching
    CORRECTING SAMPLING BIAS IN NON-MARKET VALUATION WITH KERNEL MEAN MATCHING Rui Zhang Department of Agricultural and Applied Economics University of Georgia [email protected] Selected Paper prepared for presentation at the 2017 Agricultural & Applied Economics Association Annual Meeting, Chicago, Illinois, July 30 - August 1 Copyright 2017 by Rui Zhang. All rights reserved. Readers may make verbatim copies of this document for non-commercial purposes by any means, provided that this copyright notice appears on all such copies. Abstract Non-response is common in surveys used in non-market valuation studies and can bias the parameter estimates and mean willingness to pay (WTP) estimates. One approach to correct this bias is to reweight the sample so that the distribution of the characteristic variables of the sample can match that of the population. We use a machine learning algorism Kernel Mean Matching (KMM) to produce resampling weights in a non-parametric manner. We test KMM’s performance through Monte Carlo simulations under multiple scenarios and show that KMM can effectively correct mean WTP estimates, especially when the sample size is small and sampling process depends on covariates. We also confirm KMM’s robustness to skewed bid design and model misspecification. Key Words: contingent valuation, Kernel Mean Matching, non-response, bias correction, willingness to pay 2 1. Introduction Nonrandom sampling can bias the contingent valuation estimates in two ways. Firstly, when the sample selection process depends on the covariate, the WTP estimates are biased due to the divergence between the covariate distributions of the sample and the population, even the parameter estimates are consistent; this is usually called non-response bias.
    [Show full text]
  • Evaluation of Selection Bias in an Internet-Based Study of Pregnancy Planners
    HHS Public Access Author manuscript Author ManuscriptAuthor Manuscript Author Epidemiology Manuscript Author . Author manuscript; Manuscript Author available in PMC 2016 April 04. Published in final edited form as: Epidemiology. 2016 January ; 27(1): 98–104. doi:10.1097/EDE.0000000000000400. Evaluation of Selection Bias in an Internet-based Study of Pregnancy Planners Elizabeth E. Hatcha, Kristen A. Hahna, Lauren A. Wisea, Ellen M. Mikkelsenb, Ramya Kumara, Matthew P. Foxa, Daniel R. Brooksa, Anders H. Riisb, Henrik Toft Sorensenb, and Kenneth J. Rothmana,c aDepartment of Epidemiology, Boston University School of Public Health, Boston, MA bDepartment of Clinical Epidemiology, Aarhus University Hospital, Aarhus, Denmark cRTI Health Solutions, Durham, NC Abstract Selection bias is a potential concern in all epidemiologic studies, but it is usually difficult to assess. Recently, concerns have been raised that internet-based prospective cohort studies may be particularly prone to selection bias. Although use of the internet is efficient and facilitates recruitment of subjects that are otherwise difficult to enroll, any compromise in internal validity would be of great concern. Few studies have evaluated selection bias in internet-based prospective cohort studies. Using data from the Danish Medical Birth Registry from 2008 to 2012, we compared six well-known perinatal associations (e.g., smoking and birth weight) in an inter-net- based preconception cohort (Snart Gravid n = 4,801) with the total population of singleton live births in the registry (n = 239,791). We used log-binomial models to estimate risk ratios (RRs) and 95% confidence intervals (CIs) for each association. We found that most results in both populations were very similar.
    [Show full text]
  • Testing for Selection Bias IZA DP No
    IZA DP No. 8455 Testing for Selection Bias Joonhwi Joo Robert LaLonde September 2014 DISCUSSION PAPER SERIES Forschungsinstitut zur Zukunft der Arbeit Institute for the Study of Labor Testing for Selection Bias Joonhwi Joo University of Chicago Robert LaLonde University of Chicago and IZA Discussion Paper No. 8455 September 2014 IZA P.O. Box 7240 53072 Bonn Germany Phone: +49-228-3894-0 Fax: +49-228-3894-180 E-mail: [email protected] Any opinions expressed here are those of the author(s) and not those of IZA. Research published in this series may include views on policy, but the institute itself takes no institutional policy positions. The IZA research network is committed to the IZA Guiding Principles of Research Integrity. The Institute for the Study of Labor (IZA) in Bonn is a local and virtual international research center and a place of communication between science, politics and business. IZA is an independent nonprofit organization supported by Deutsche Post Foundation. The center is associated with the University of Bonn and offers a stimulating research environment through its international network, workshops and conferences, data service, project support, research visits and doctoral program. IZA engages in (i) original and internationally competitive research in all fields of labor economics, (ii) development of policy concepts, and (iii) dissemination of research results and concepts to the interested public. IZA Discussion Papers often represent preliminary work and are circulated to encourage discussion. Citation of such a paper should account for its provisional character. A revised version may be available directly from the author. IZA Discussion Paper No.
    [Show full text]
  • Correcting Sample Selection Bias by Unlabeled Data
    Correcting Sample Selection Bias by Unlabeled Data Jiayuan Huang Alexander J. Smola Arthur Gretton School of Computer Science NICTA, ANU MPI for Biological Cybernetics Univ. of Waterloo, Canada Canberra, Australia T¨ubingen, Germany [email protected] [email protected] [email protected] Karsten M. Borgwardt Bernhard Scholkopf¨ Ludwig-Maximilians-University MPI for Biological Cybernetics Munich, Germany T¨ubingen, Germany [email protected]fi.lmu.de [email protected] Abstract We consider the scenario where training and test data are drawn from different distributions, commonly referred to as sample selection bias. Most algorithms for this setting try to first recover sampling distributions and then make appro- priate corrections based on the distribution estimate. We present a nonparametric method which directly produces resampling weights without distribution estima- tion. Our method works by matching distributions between training and testing sets in feature space. Experimental results demonstrate that our method works well in practice. 1 Introduction The default assumption in many learning scenarios is that training and test data are independently and identically (iid) drawn from the same distribution. When the distributions on training and test set do not match, we are facing sample selection bias or covariate shift. Specifically, given a domain of patterns X and labels Y, we obtain training samples Z = (x , y ),..., (x , y ) X Y from { 1 1 m m } ⊆ × a Borel probability distribution Pr(x, y), and test samples Z′ = (x′ , y′ ),..., (x′ ′ , y′ ′ ) X Y { 1 1 m m } ⊆ × drawn from another such distribution Pr′(x, y). Although there exists previous work addressing this problem [2, 5, 8, 9, 12, 16, 20], sample selection bias is typically ignored in standard estimation algorithms.
    [Show full text]