DOCSLIB.ORG

Arxiv:1912.11078V2 [Cs.CL] 12 Sep 2020

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Arxiv:1912.11078V2 [Cs.CL] 12 Sep 2020 Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview Deven Shah H. Andrew Schwartz Dirk Hovy Stony Brook University Bocconi University {dsshah,has}@cs.stonybrook.edu [email protected] Abstract the training process. In the case of domains, these non-generalizations are words, phrases, or senses An increasing number of natural language that occur in one text type, but not another. processing papers address the effect of bias However, this kind of variation is not just re- on predictions, introducing mitigation tech- niques at different parts of the standard NLP stricted to text domains: it is a fundamental prop- pipeline (data and models). However, these erty of human-generated language: we talk differ- works have been conducted individually, with- ently than our parents or people from a different out a unifying framework to organize efforts part of our country, etc. (Pennebaker and Stone, within the field. This situation leads to repet- 2003; Eisenstein et al., 2010; Kern et al., 2016). itive approaches, and focuses overly on bias In other words, language reflects the diverse de- symptoms/effects origins , rather than on their , mographics, backgrounds, and personalities of the which could limit the development of effective countermeasures. In this paper, we propose a people who use it. While these differences are of- unifying predictive bias framework for NLP. ten subtle, they are distinct and cumulative (Trudg- We summarize the NLP literature and suggest ill, 2000; Kern et al., 2016; Pennebaker, 2011). general mathematical definitions of predictive Similar to text domains, this variation can lead bias. We differentiate two consequences of models to pick up on patterns that do not gener- bias: outcome disparities and error dispari- alize to other author-demographics, or to rely on ties, as well as four potential origins of bi- undesirable word-demographic relationships. ases: label bias, selection bias, model over- Bias may be an inherent property of any NLP amplification, and semantic bias. Our frame- work serves as an overview of predictive bias system (and broadly any statistical model), but in NLP, integrating existing work into a single this is not per se negative. In essence, biases are structure, and providing a conceptual baseline priors that inform our decisions (a dialogue sys- for improved frameworks. tem designed for elders might work differently than one for teenagers). Still, undetected and 1 Introduction unaddressed, biases can lead to negative conse- Predictive models in NLP are sensitive to a variety quences: There are aggregate effects for demo- of (often unintended) biases throughout the devel- graphic groups, which combine to produce predic- opment process. As a result, fitted models do not tive bias. I.e., the label distribution of a predictive arXiv:1912.11078v2 [cs.CL] 12 Sep 2020 generalize well, incurring performance and relia- model reflects a human attribute in a way that di- bility losses on unseen data. They also have so- verges from a theoretically defined “ideal distribu- cially undesirable effects by systematically under- tion.” For example, a Part Of Speech (POS) tag- serving or mispredicting certain user groups. ger reflecting how an older generation uses words The general phenomenon of biased predictive (Hovy and Søgaard, 2015) diverges from the pop- models in NLP is not recent. The community ulation as a whole. has long worked on the domain adaptation prob- A variety of papers have begun to address lem (Jiang and Zhai, 2007; Daume III, 2007): countermeasures for predictive biases (Li et al., models fit on newswire data do not perform well 2018; Elazar and Goldberg, 2018; Coavoux et al., on social media and other text types. This prob- 2018).1 Each identifies a specific bias and counter- lem arises from the tendency of statistical mod- 1An even more extensive body of work on fairness exists els to pick up on non-generalizable signals during as part of the FAT* conferences, which goes beyond the scope Error disparity Example Dep, is_older? Matched / uniform: outcome - 0 (young) : .001 origin over-amplification label bias outcome disparity ideal: - 1 (old) : .001 The model discriminates on Biased annotations, The distribution of outcomes, given attribute A, 0, 0 : .30 consequence a given human attribute interaction, or latent bias is dissimilar than the ideal distribution: 0, 1 : .35 From predicted beyond its source base-rate. from past classifications. Q(Ŷ |A) ≁ P(Y |A) t t 1, 0 : .20 - 0 (young) : 1, 1 : .15 mean=.0005 Embedding - 1 (old) : Corpus Source Population Target Population From predicted mean=.002 biased 0, 0 : .25 features features outcomes features 0, 1 : .40 Q(ϵ |A) ≁ Uniform => there fit predict outcomes t 휃 X Y X 1, 0 : .25 exists an i,j such that embedding source source target Ŷ target ϵ ϵ (Pre-trained Side) (Model Side) (Application Side) 1, 1 : .20 Q( t|Ai) =/= Q( t|Aj) semantic bias selection bias error disparity Non-ideal associations between attributed The sample of observations The distribution of error (ϵ) over at least two lexeme (e.g. gendered pronouns) and themselves are not representative different values of an attribute A( ) are unequal: non-attributed lexeme (e.g. occupation). of the application population. ϵ ϵ Q( t|Ai) ≁ Q( t|Aj) Figure 1: The Predictive Bias Framework for NLP: Depiction of where bias mayerror originate disparity within a standard supervised NLP pipeline. Evidence of bias is seen in y^ via outcome disparityTheand distributionerror disparityof error (ϵ) .is inconsistent over different values of an attribute A( ): ϵ Q( t|A) ≁ Uniform measure on their terms, but it is often not explic- lected works in social science and adjacent fields. itly clear which bias is addressed, where it orig- We identify four distinct sources of bias: selec- inates, or how it generalizes. There are multi- tion bias, label bias, model overamplification, ple sources from which bias can arise within the and semantic bias. We can express all of these as predictive pipeline, and methods proposed for one differences between (a) a “true” or intended distri- specific bias often do not apply to another. As bution (e.g., over users, labels, or outcomes), and a consequence, much work has focused on bias (b) the distribution used or produced by the model. effects and symptoms rather than their origins. These cases arise at specific points within a typical While it is essential to address the effects of bias, it predictive pipeline: embeddings, source data, la- can leave the fundamental origin unchanged (Go- bels (human annotators), models, and target data. nen and Goldberg, 2019), requiring researchers to We provide quantitative definitions of predictive rediscover the issue over and over. The “bias” dis- bias in this framework intended to make it easier cussed in one paper may, therefore, be quite dif- to: (a) identify biases (because they can be clas- ferent than that in another.2 sified), (b) develop countermeasures (because the A shared definition and framework of predictive underlying problem is known), and (c) compare bias can unify these efforts, provide a common ter- biases and countermeasures across papers. We minology, help to identify underlying causes, and hope this paper will help researchers spot, com- allow coordination of countermeasures (Sun et al., pare, and address bias in all its various forms. 2019). However, such a general framework had Contributions Our primary contributions in- yet to be proposed within the NLP community. clude: (1) a conceptual framework for identify- To address these problems, we suggest a joint ing and quantifying predictive bias and its origins conceptual framework, depicted in Figure1, out- within a standard NLP pipeline, (2) a survey of bi- lining and relating the different origins of bias. ases identified in NLP models, and (3) a survey We base our framework on an extensive survey of methods for countering bias in NLP organized of the relevant NLP literature, informed by se- within our conceptual framework. of this biased-focused paper. Note also that while bias is an 2 Definition - Two Types of Disparities ethical issue and contributes to many papers in the ethics in NLP area, the two should not be conflated: ethics covers more Our definition of predictive bias in NLP builds on than bias. 2Quantitative social science offers a background for its definition within the literature on standardized bias (Berk, 1983). However, NLP differs fundamentally in testing (i.e., SAT, GRE, etc.) Specifically, Swinton analytic goals (namely out-of-sample prediction for NLP ver- (1981) states: sus parameter inference for hypothesis testing in social sci- ence) that bring about NLP-specific situations: biases in word embeddings, annotator labels, or predicting over-amplified By “predictive bias," we refer to a situation in demographics. which a [predictive model] is used to predict a specific criterion for a particular population, and pronouns associated with computer science). Our is found to give systematically different predic- framework should enable its users to apply evolv- tions for subgroups of this population who are in ing standards and norms across NLP’s many ap- fact identical on that specific criterion.3 plication contexts. A prototypical example of outcome disparity We generalize Swinton’s definition in two ways: is gender disparity in image captions. Zhao et al. First, to align notation with standard supervised (2017) and Hendricks et al.(2018) demonstrate modeling, we say there are both Y (a random a systematic difference with respect to gender in variable representing the “true” values of an out- the outcome of the model, Y^ even when taking come) and Y^ (a random variable representing the the source distribution as an ideal target distribu- predictions. Next, we allow the concept to apply tion: Q(Y^ jgender) Q(Y jgender) ∼ to differences associated with continuously-valued target target Q(Y jgender).
Load more

Recommended publications
  • Applying Library Values to Emerging Technology Decision-Making in the Age of Open Access, Maker Spaces, and the Ever-Changing Library
    ACRL Publications in Librarianship No. 72 Applying Library Values to Emerging Technology Decision-Making in the Age of Open Access, Maker Spaces, and the Ever-Changing Library Editors Peter D. Fernandez and Kelly Tilton Association of College and Research Libraries A division of the American Library Association Chicago, Illinois 2018 The paper used in this publication meets the minimum requirements of Ameri- can National Standard for Information Sciences–Permanence of Paper for Print- ed Library Materials, ANSI Z39.48-1992. ∞ Cataloging-in-Publication data is on file with the Library of Congress. Copyright ©2018 by the Association of College and Research Libraries. All rights reserved except those which may be granted by Sections 107 and 108 of the Copyright Revision Act of 1976. Printed in the United States of America. 22 21 20 19 18 5 4 3 2 1 Contents Contents Introduction .......................................................................................................ix Peter Fernandez, Head, LRE Liaison Programs, University of Tennessee Libraries Kelly Tilton, Information Literacy Instruction Librarian, University of Tennessee Libraries Part I Contemplating Library Values Chapter 1. ..........................................................................................................1 The New Technocracy: Positioning Librarianship’s Core Values in Relationship to Technology Is a Much Taller Order Than We Think John Buschman, Dean of University Libraries, Seton Hall University Chapter 2. ........................................................................................................27
    [Show full text]
  • Recognizing and Overcoming Bias
    Recognizing and Overcoming Bias Ways to Make Your Workplace More Successful & Welcoming Sheeri Kritzer Cabral, Mozilla Leslie Hawthorn, Elasticsearch USENIX LISA 2013 Introductions - Leslie ● Community Manager, Elasticsearch ● Worked in high tech field for 10+ years ● Interests include helping newbies and humanitarian open source software Introductions - Sheeri ● Volunteer activist in a previous life ● Fulltime as a sysadmin, DBA since 2001 ● Encourages learning cooperatively, not competitively Reality Check We’ll be standing up here for about an hour talking about bias …. We’re both also well-educated, well paid white women working in the tech industry. We have a whole lot to learn about how bias operates in the workplace and our wider world. The Ground Rules We’ll be discussing some difficult and contentious topics for the next three hours. ● Create a safe environment ○ Listen without judgement ○ Disclose without fear ○ Disagree respectfully ● If we cannot be candid with and amongst ourselves, we cannot improve our situation Around the Room Please take a moment to ● Introduce yourself ● Tell us why you chose to come to this workshop ● Let everyone know what you hope to learn by participating Agenda 9:00 - 9:20 AM: Introductions - all 9:20 - 10:30 AM: Exploring Bias + Ways to Overcome It - Sheeri and Leslie 10:30 - 10:45 AM: Break 10:45 - 11:30 AM: Small Group Discussions - all 11:30 - 11:45 AM: Report Back from Breakouts - all 11:45 AM - 12 Noon: Break 12 Noon - 12:30 PM: Leveling Up - Sheeri and Leslie Understanding Bias Exploring
    [Show full text]
  • 2018 Ybarra Vincent T Thesis.Pdf (1.288Mb)
    UNIVERSITY OF OKLAHOMA GRADUATE COLLEGE SELF-EVALUATION OF SKILLS AND OVERCONFIDENCE VULNERABILITY: ARE MOST PEOPLE BLIND TO THEIR OWN DECISION MAKING BIASES? A THESIS SUBMITTED TO THE GRADUATE FACULTY in partial fulfillment of the requirements for the Degree of MASTER OF SCIENCE By VINCENT T. YBARRA Norman, Oklahoma 2018 SELF-EVALUATION OF SKILL AND OVERCONFIDENCE VULNERABILITY: ARE MOST PEOPLE BLIND TO THEIR DECISION MAKING BIASES? A THESIS APPROVED FOR THE DEPARTMENT OF PSYCHOLOGY BY ______________________________ Dr. Edward Cokely, Chair ______________________________ Dr. Robert Terry ______________________________ Dr. Scott Gronlund ______________________________ Dr. Rocio Garcia-Retamero © Copyright by VINCENT T. YBARRA 2018 All Rights Reserved. Dedicated to my grandfather Robert J. Brunel Table of Contents List of Tables ................................................................................................................... vi List of Figures ................................................................................................................. vii Abstract .......................................................................................................................... viii Chapter 1: Numeracy and Self-Evaluation of Decision Skill ........................................... 1 1.1 An Overview of the Bias Blind Spot .................................................................... 1 1.2 Bias Blind Spot and How it Relates to General Decision Making Skill .............. 8 Chapter 2: Participants and
    [Show full text]
  • Hot-Hand Bias in Rhesus Monkeys
    Journal of Experimental Psychology: Animal Learning and Cognition © 2014 American Psychological Association 2014, Vol. 40, No. 3, 280–286 2329-8456/14/$12.00 http://dx.doi.org/10.1037/xan0000033 BRIEF REPORT Hot-Hand Bias in Rhesus Monkeys Tommy C. Blanchard Andreas Wilke University of Rochester Clarkson University Benjamin Y. Hayden University of Rochester Human decision-makers often exhibit the hot-hand phenomenon, a tendency to perceive positive serial autocorrelations in independent sequential events. The term is named after the observation that basketball fans and players tend to perceive streaks of high accuracy shooting when they are demonstrably absent. That is, both observing fans and participating players tend to hold the belief that a player’s chance of hitting a shot are greater following a hit than following a miss. We hypothesize that this bias reflects a strong and stable tendency among primates (including humans) to perceive positive autocorrelations in temporal sequences, that this bias is an adaptation to clumpy foraging environments, and that it may even be ecologically rational. Several studies support this idea in humans, but a stronger test would be to determine whether nonhuman primates also exhibit a hot-hand bias. Here we report behavior of 3 monkeys performing a novel gambling task in which correlation between sequential gambles (i.e., temporal clumpiness) is systematically manipulated. We find that monkeys have better performance (meaning, more optimal behavior) for clumped (positively correlated) than for dispersed (negatively correlated) distributions. These results identify and quantify a new bias in monkeys’ risky decisions, support accounts that specifically incorporate cognitive biases into risky choice, and support the suggestion that the hot-hand phenomenon is an evolutionary ancient bias.
    [Show full text]
  • School of National Security, Intelligence
    A Critical Thinking Primer An Unpublished Manuscript Prepared by: Michael W. Collier, Ph.D. [email protected] Revised: January 2019 © Michael W. Collier 2019 Table of Contents Section Title Pages Introduction 2 The Vocabulary of Research & Analysis 3-10 A Critical Thinking Framework 10-12 Why Critical Thinking? 12-18 Elements: Purpose and Questions 18-21 Element: Information (What We Already Know) 21-29 Elements: Points of View (Perspectives) & Assumptions 30-33 Perspectives & Assumptions: Cultural Assessments 34-39 Perspectives & Assumptions: Theoretical Assessments 39-58 Element: Developing Alternatives 58-63 Measuring Variables & Sampling 64-70 Element: Interpretation/Inference (Qualitative Analysis) 71-81 Research and Analysis Standards 82-85 References 86-88 Appendix 1: A Critical Thinking Framework for Security 89-95 Analysis Appendix 2: Security Analysis: Written Reports and Verbal 96-123 Briefings (includes final section on Logic Fallacies) 1 Introduction Welcome to your course! This manuscript provides the basics of critical thinking you may use in your personal and professional lives. The skills required to conduct critical thinking are the most in-depth of the many types of research and analysis you will experience in your personal lives and most professional careers. This manuscript focuses mainly on blending critical thinking techniques with social science research methods and procedures—skills you can then adapt and use in other types of personal and professional analyses. This manuscript varies somewhat from mainstream social science research methods material by adopting a framework for research, analysis, and writing/presenting based largely in critical thinking and creative thinking techniques. You may have been introduced previously to critical thinking and critical reading material.
    [Show full text]
  • Science Based Human Reliability Analysis: Using Digital Nuclear Power Plant Simulators for Human Reliability Research
    Science Based Human Reliability Analysis: Using Digital Nuclear Power Plant Simulators for Human Reliability Research DISSERTATION Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the Graduate School of The Ohio State University By Rachel Elizabeth Shirley Graduate Program in Nuclear Engineering The Ohio State University 2017 Dissertation Committee: Dr. Carol Smidts, Advisor Dr. Tunc Aldemir Dr. Ronald Boring Dr. Catherine Calder Copyrighted by Rachel Elizabeth Shirley 2017 Abstract Nuclear power plant (NPP) simulators are proliferating in academic research institutions and national laboratories in response to the availability of affordable, digital simulator platforms. Accompanying the new research facilities is a renewed interest in using data collected in NPP simulators for Human Reliability Analysis (HRA) research. An experiment conducted in The Ohio State University (OSU) NPP Simulator Facility develops data collection methods and analytical tools to improve use of simulator data in HRA. In the pilot experiment, student operators respond to design basis accidents in the OSU NPP Simulator Facility. Thirty-three undergraduate and graduate engineering students participated in the research. Following each accident scenario, student operators completed a survey about perceived simulator biases and watched a video of the scenario. During the video, they periodically recorded their perceived strength of significant Performance Shaping Factors (PSFs) such as Stress. This dissertation reviews three aspects of simulator-based research using the data collected in the OSU NPP Simulator Facility: First, a qualitative comparison of student operator performance to computer simulations of expected operator performance generated by the Information Decision Action Crew ii (IDAC) HRA method. Areas of comparison include procedure steps, timing of operator actions, and PSFs.
    [Show full text]
  • Assessment of Factors Causing Bias in Marketing- Related Publications
    publications Article Assessment of Factors Causing Bias in Marketing- Related Publications Mangirdas Morkunas 1,*, Elze˙ Rudiene˙ 2, Lukas Giriunas¯ 3 and Laura Dauˇciunien¯ e˙ 3 1 Division of Farms’ and Enterprises’ Economics, Lithuanian Institute of Agrarian Economics, Vivulskio str. 4A, 03220 Vilnius, Lithuania 2 Business School, Vilnius University, Sauletekio ave. 21, 10222 Vilnius, Lithuania; [email protected] 3 Faculty of Public Governance and Business, Mykolas Romeris university, Ateities str. 20, 08303 Vilnius, Lithuania; [email protected] (L.G.); [email protected] (L.D.) * Correspondence: [email protected]; Received: 13 September 2020; Accepted: 15 October 2020; Published: 24 October 2020 Abstract: The present paper aims at revealing and ranking the factors that most frequently cause bias in marketing-related publications. In order to rank the factors causing bias, the authors employed the Analytic Hierarchy Process method with three different scales representing all scale groups. The data for the study were obtained through expert survey, which involved nine experts both from the academia and scientific publishing community. The findings of the study confirm that factors that most frequently cause bias in marketing related publications are sampling and sample frame errors, failure to specify the inclusion and exclusion criteria for researched subjects and non-responsiveness. Keywords: bias; research; marketing; publication 1. Introduction Bias can be defined as any systematic error in the design, conduct or analysis of a study. In research, bias occurs when “systematic error introduced into sampling or testing by selecting or encouraging one outcome or answer over others”. Bias can attend at any phase of research, including study design or data collection, as well as in the process of data analysis and publication.
    [Show full text]
  • Intelligence Testing and Cultural Diversity: Concerns, Cautions, and Considerations
    THE NATIONAL RESEARCH CENTER NRC ON THE GIFTED G/T AND TALENTED Senior Scholars Series University of Connecticut University of Virginia Yale University Intelligence Testing and Cultural Diversity: Concerns, Cautions, and Considerations Donna Y. Ford Vanderbilt University Nashville, Tennessee December 2004 RM04204 Intelligence Testing and Cultural Diversity: Concerns, Cautions, and Considerations Donna Y. Ford Vanderbilt University Nashville, Tennessee December 2004 RM04204 THE NATIONAL RESEARCH CENTER ON THE GIFTED AND TALENTED The National Research Center on the Gifted and Talented (NRC/GT) is funded under the Jacob K. Javits Gifted and Talented Students Education Act, Institute of Education Sciences, United States Department of Education. The Directorate of the NRC/GT serves as an administrative and a research unit and is located at the University of Connecticut. The participating universities include the University of Virginia and Yale University, as well as a research unit at the University of Connecticut. University of Connecticut Dr. Joseph S. Renzulli, Director Dr. E. Jean Gubbins, Associate Director Dr. Sally M. Reis, Associate Director University of Virginia Dr. Carolyn M. Callahan, Associate Director Yale University Dr. Robert J. Sternberg, Associate Director Copies of this report are available from: NRC/GT University of Connecticut 2131 Hillside Road Unit 3007 Storrs, CT 06269-3007 Visit us on the web at: www.gifted.uconn.edu The work reported herein was supported under the Educational Research and Development Centers Program, PR/Award Number R206R000001, as administered by the Institute of Education Sciences, U.S. Department of Education. The findings and opinions expressed in this report do not reflect the position or policies of the Institute of Education Sciences or the U.S.
    [Show full text]
  • Gender Bias in the Job Market: a Longitudinal Analysis
    Gender Bias in the Job Market: A Longitudinal Analysis SHILIANG TANG, UC Santa Barbara XINYI ZHANG, UC Santa Barbara JENNA CRYAN, University of Chicago MIRIAM J. METZGER, UC Santa Barbara HAITAO ZHENG, University of Chicago BEN Y. ZHAO, University of Chicago For millions of workers, online job listings provide the first point of contact to potential employers. Asa result, job listings and their word choices can significantly affect the makeup of the responding applicant pool. Here, we study the effects of potentially gender-biased terminology in job listings, and their impact onjob 99 applicants, using a large historical corpus of 17 million listings on LinkedIn spanning 10 years. We develop algorithms to detect and quantify gender bias, validate them using external tools, and use them to quantify job listing bias over time. We then perform a user survey over two user populations (N1=469, N2=273) to validate our findings and to quantify the end-to-end impact of such bias on applicant decisions. Our findings show gender-bias has decreased significantly over the last 10 years. More surprisingly, we find that impact ofgender bias in listings is dwarfed by our respondents’ inherent bias towards specific job types. CCS Concepts: • Human-centered computing → Computer supported cooperative work; • Social and professional topics → Employment issues; • Applied computing → Sociology; Additional Key Words and Phrases: Gender Bias, Online Job Advertisements, Quantitative Analysis, User Studies ACM Reference format: Shiliang Tang, Xinyi Zhang, Jenna Cryan, Miriam J. Metzger, Haitao Zheng, and Ben Y. Zhao. 2017. Gender Bias in the Job Market: A Longitudinal Analysis. Proc. ACM Hum.-Comput.
    [Show full text]
  • Big Data in Organizations and the Role of Human Resource Management PERSONALMANAGEMENT UND ORGANISATION Herausgegeben Von Volker Stein
    5 Big data are changing the way we work as companies face an increas- ing amount of data. Rather than replacing a human workforce or making Tobias M. Scholz decisions obsolete, big data are going to pose an immense innovating force to those employees capable of utilizing them. This book intends to first convey a theoretical understanding of big data. It then tackles the phenomenon of big data from the perspectives of varied organizational theories in order to highlight socio-technological interaction. Big data are Big Data bound to transform organizations which calls for a transformation of the human resource department. The HR department’s new role then enables organizations to utilize big data for their purpose. Employees, while re- maining an organization’s major competitive advantage, have found a in Organizations powerful ally in big data. and the Role of Human Resource Management A Complex Systems Theory-Based Conceptualization · Big Data in Organizations and the Role of HRM Tobias M. Scholz is currently holding a position as a Post-Doctoral Re- searcher at the University of Siegen. After graduating from universities in Germany and the U.S., he has worked as a Research and Teaching Assistant. His field of research is human resource management and or- Umschlaggestaltung: ganizational behavior. © Olaf Gloeckler, Atelier Platen, Friedberg Tobias M. Scholz Tobias PERSONALMANAGEMENT UND ORGANISATION Cover Design: Herausgegeben von Volker Stein© Olaf Gloeckler, Atelier Platen, Friedberg www.peterlang.com ISBN 978-3-631-71890-2 Conception de la couverture du livre: © Olaf Gloeckler, Atelier Platen, Friedberg PUO 05_271890_Scholz_AM_A5HCk PLA.indd 1 02.03.17 KW 09 15:23 5 Big data are changing the way we work as companies face an increas- ing amount of data.
    [Show full text]
  • Addressing Onsite Sampling in Recreation Demand Models
    Addressing Onsite Sampling in Recreation Site Choice Models Paul Hindsley,1* Craig E. Landry,2 and Brad Gentner3 1Environmental Studies, Eckerd College, St. Petersburg, FL 33711 [email protected] 2 Department of Economics, East Carolina University, Greenville, NC 27858 [email protected] 3Gentner Consulting Group, Silver Spring, MD 20901 [email protected] Abstract Independent experts and politicians have criticized statistical analyses of recreation behavior that rely upon onsite samples due to their potential for biased inference, prompting some to suggest support for these efforts should be curtailed. The use of onsite sampling usually reflects data or budgetary constraints but can lead to two primary forms of bias in site choice models. First, the strategy entails sampling site choices rather than sampling anglers– a form of bias called endogenous stratification. Under these conditions, sample choices may not reflect the site choices of the true population. Second, the exogenous attributes of the recreational users sampled onsite may differ from the attributes of users in the population – the most common form in recreation demand is avidity bias. We propose addressing these biases by combining two existing methods, Weighted Exogenous Stratification Maximum Likelihood Estimation (WESMLE) and propensity score estimation. We use the National Marine Fisheries Service’s (NMFS) Marine Recreational Fishing Statistics Survey (MRFSS) to illustrate methods of bias reduction employing both simulation and empirical applications. We find that propensity score based weights can significantly reduce bias in estimation. Our results indicate that failure to account for these biases can overstate anglers’ willingness to pay for additional fishing catch. Key words: On-site sampling, propensity score weighting, recreation demand, random utility models * Corresponding Author.
    [Show full text]
  • Can AI Solve the Diversity Problem in the Tech Industry? Mitigating Noise and Bias in Employment Decision-Making
    Can AI Solve the Diversity Problem in the Tech Industry? Mitigating Noise and Bias in Employment Decision-Making Kimberly A. Houser*1 22 STAN. TECH. L. REV. 290 (2019) ABSTRACT After the first diversity report was issued in 2014 revealing the dearth of women in the tech industry, companies rushed to hire consultants to provide unconscious bias training to their employees. Unfortunately, recent diversity reports show no significant improvement, and, in fact, women lost ground during some of the years. According to a Human Capital Institute survey, nearly 80% of leaders were still using gut feeling and personal opinion to make decisions that affected talent-management practices. By incorporating AI into employment decisions, we can mitigate unconscious bias and variability (noise) in human decision-making. While some scholars have warned that using artificial intelligence (AI) in decision-making creates discriminatory results, they downplay the reason for such occurrences— humans. The main concerns noted relate to the risk of reproducing bias in an algorithmic outcome (“garbage in, garbage out”) and the inability to detect bias due to the lack of understanding of the reason for the algorithmic outcome (“black box” problem). In this paper, I argue that responsible AI will abate the problems caused by unconscious biases and noise in human decision- making, and in doing so increase the hiring, promotion, and retention of women in the tech industry. The new solutions to the garbage in, garbage out and black box concerns will be explored. The question is not whether AI should * Kimberly A. Houser is an assistant professor at Oklahoma State University.
    [Show full text]