Reproducibility and Privacy: Research Data Use, Reuse, and Abuse
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Conjunction Fallacy' Revisited: How Intelligent Inferences Look Like Reasoning Errors
Journal of Behavioral Decision Making J. Behav. Dec. Making, 12: 275±305 (1999) The `Conjunction Fallacy' Revisited: How Intelligent Inferences Look Like Reasoning Errors RALPH HERTWIG* and GERD GIGERENZER Max Planck Institute for Human Development, Berlin, Germany ABSTRACT Findings in recent research on the `conjunction fallacy' have been taken as evid- ence that our minds are not designed to work by the rules of probability. This conclusion springs from the idea that norms should be content-blind Ð in the present case, the assumption that sound reasoning requires following the con- junction rule of probability theory. But content-blind norms overlook some of the intelligent ways in which humans deal with uncertainty, for instance, when drawing semantic and pragmatic inferences. In a series of studies, we ®rst show that people infer nonmathematical meanings of the polysemous term `probability' in the classic Linda conjunction problem. We then demonstrate that one can design contexts in which people infer mathematical meanings of the term and are therefore more likely to conform to the conjunction rule. Finally, we report evidence that the term `frequency' narrows the spectrum of possible interpreta- tions of `probability' down to its mathematical meanings, and that this fact Ð rather than the presence or absence of `extensional cues' Ð accounts for the low proportion of violations of the conjunction rule when people are asked for frequency judgments. We conclude that a failure to recognize the human capacity for semantic and pragmatic inference can lead rational responses to be misclassi®ed as fallacies. Copyright # 1999 John Wiley & Sons, Ltd. KEY WORDS conjunction fallacy; probabalistic thinking; frequentistic thinking; probability People's apparent failures to reason probabilistically in experimental contexts have raised serious concerns about our ability to reason rationally in real-world environments. -
Thinking About the Ultimate Argument for Realism∗
Thinking About the Ultimate Argument for Realism∗ Stathis Psillos Department of Philosophy and History of Science University of Athens Panepistimioupolis (University Campus) Athens 15771 Greece [email protected] 1. Introduction Alan Musgrave has been one of the most passionate defenders of scientific realism. Most of his papers in this area are, by now, classics. The title of my paper alludes to Musgrave’s piece “The Ultimate Argument for Realism”, though the expression is Bas van Fraassen’s (1980, 39), and the argument is Hilary Putnam’s (1975, 73): realism “is the only philosophy of science that does not make the success of science a miracle”. Hence, the code-name ‘no-miracles’ argument (henceforth, NMA). In fact, NMA has quite a history and a variety of formulations. I have documented all this in my (1999, chapter 4). But, no matter how exactly the argument is formulated, its thrust is that the success of scientific theories lends credence to the following two theses: a) that scientific theories should be interpreted realistically and b) that, so interpreted, these theories are approximately true. The original authors of the argument, however, did not put an extra stress on novel predictions, which, as Musgrave (1988) makes plain, is the litmus test for the ability of any approach to science to explain the success of science. Here is why reference to novel predictions is crucial. Realistically understood, theories entail too many novel claims, most of them about unobservables (e.g., that ∗ I want to dedicate this paper to Alan Musgrave. His exceptional combination of clear-headed and profound philosophical thinking has been a model for me. -
Graphical Techniques in Debiasing: an Exploratory Study
GRAPHICAL TECHNIQUES IN DEBIASING: AN EXPLORATORY STUDY by S. Bhasker Information Systems Department Leonard N. Stern School of Business New York University New York, New York 10006 and A. Kumaraswamy Management Department Leonard N. Stern School of Business New York University New York, NY 10006 October, 1990 Center for Research on Information Systems Information Systems Department Leonard N. Stern School of Business New York University Working Paper Series STERN IS-90-19 Forthcoming in the Proceedings of the 1991 Hawaii International Conference on System Sciences Center for Digital Economy Research Stem School of Business IVorking Paper IS-90-19 Center for Digital Economy Research Stem School of Business IVorking Paper IS-90-19 2 Abstract Base rate and conjunction fallacies are consistent biases that influence decision making involving probability judgments. We develop simple graphical techniques and test their eflcacy in correcting for these biases. Preliminary results suggest that graphical techniques help to overcome these biases and improve decision making. We examine the implications of incorporating these simple techniques in Executive Information Systems. Introduction Today, senior executives operate in highly uncertain environments. They have to collect, process and analyze a deluge of information - most of it ambiguous. But, their limited information acquiring and processing capabilities constrain them in this task [25]. Increasingly, executives rely on executive information/support systems for various purposes like strategic scanning of their business environments, internal monitoring of their businesses, analysis of data available from various internal and external sources, and communications [5,19,32]. However, executive information systems are, at best, support tools. Executives still rely on their mental or cognitive models of their businesses and environments and develop heuristics to simplify decision problems [10,16,25]. -
Downloading of a Human Consciousness Into a Digital Computer Would Involve ‘A Certain Loss of Our Finer Feelings and Qualities’89
When We Can Trust Computers (and When We Can’t) Peter V. Coveney1,2* Roger R. Highfield3 1Centre for Computational Science, University College London, Gordon Street, London WC1H 0AJ, UK, https://orcid.org/0000-0002-8787-7256 2Institute for Informatics, Science Park 904, University of Amsterdam, 1098 XH Amsterdam, Netherlands 3Science Museum, Exhibition Road, London SW7 2DD, UK, https://orcid.org/0000-0003-2507-5458 Keywords: validation; verification; uncertainty quantification; big data; machine learning; artificial intelligence Abstract With the relentless rise of computer power, there is a widespread expectation that computers can solve the most pressing problems of science, and even more besides. We explore the limits of computational modelling and conclude that, in the domains of science and engineering that are relatively simple and firmly grounded in theory, these methods are indeed powerful. Even so, the availability of code, data and documentation, along with a range of techniques for validation, verification and uncertainty quantification, are essential for building trust in computer generated findings. When it comes to complex systems in domains of science that are less firmly grounded in theory, notably biology and medicine, to say nothing of the social sciences and humanities, computers can create the illusion of objectivity, not least because the rise of big data and machine learning pose new challenges to reproducibility, while lacking true explanatory power. We also discuss important aspects of the natural world which cannot be solved by digital means. In the long-term, renewed emphasis on analogue methods will be necessary to temper the excessive faith currently placed in digital computation. -
Caveat Emptor: the Risks of Using Big Data for Human Development
1 Caveat emptor: the risks of using big data for human development Siddique Latif1,2, Adnan Qayyum1, Muhammad Usama1, Junaid Qadir1, Andrej Zwitter3, and Muhammad Shahzad4 1Information Technology University (ITU)-Punjab, Pakistan 2University of Southern Queensland, Australia 3University of Groningen, Netherlands 4National University of Sciences and Technology (NUST), Pakistan Abstract Big data revolution promises to be instrumental in facilitating sustainable development in many sectors of life such as education, health, agriculture, and in combating humanitarian crises and violent conflicts. However, lurking beneath the immense promises of big data are some significant risks such as (1) the potential use of big data for unethical ends; (2) its ability to mislead through reliance on unrepresentative and biased data; and (3) the various privacy and security challenges associated with data (including the danger of an adversary tampering with the data to harm people). These risks can have severe consequences and a better understanding of these risks is the first step towards mitigation of these risks. In this paper, we highlight the potential dangers associated with using big data, particularly for human development. Index Terms Human development, big data analytics, risks and challenges, artificial intelligence, and machine learning. I. INTRODUCTION Over the last decades, widespread adoption of digital applications has moved all aspects of human lives into the digital sphere. The commoditization of the data collection process due to increased digitization has resulted in the “data deluge” that continues to intensify with a number of Internet companies dealing with petabytes of data on a daily basis. The term “big data” has been coined to refer to our emerging ability to collect, process, and analyze the massive amount of data being generated from multiple sources in order to obtain previously inaccessible insights. -
Zusammenfassung
Zusammenfassung Auf den folgenden beiden Seiten werden die zuvor detailliert darge- stellten 100 Fehler nochmals übersichtlich gruppiert und klassifiziert. Der kompakte Überblick soll Sie dabei unterstützen, die systemati- schen Fehler durch geeignete Maßnahmen möglichst zu verhindern beziehungsweise so zu reduzieren, dass sich keine allzu negativen Auswirkungen auf Ihr operatives Geschäft ergeben! Eine gezielte und strukturierte Auseinandersetzung mit den Fehler(kategorie)n kann Sie ganz allgemein dabei unterstützen, Ihr Unternehmen „resilienter“ und „antifragiler“ aufzustellen, das heißt, dass Sie von negativen Entwicklungen nicht sofort umgeworfen werden (können). Außerdem kann eine strukturierte Auseinander- setzung auch dabei helfen, sich für weitere, hier nicht explizit genannte – aber ähnliche – Fehler zu wappnen. © Springer Fachmedien Wiesbaden GmbH, ein Teil von Springer Nature 2019 402 C. Glaser, Risiko im Management, https://doi.org/10.1007/978-3-658-25835-1 403 Zusammenfassung Zu viele Zu wenig Nicht ausreichend Informationen Informationsgehalt Zeit zur Verfügung Deshalb beachten wir Deshalb füllen wir die Deshalb nehmen wir häufig typischerweise nur… Lücken mit… an, dass… Veränderungen Mustern und bekannten wir Recht haben Außergewöhnliches Geschichten wir das schaffen können Wiederholungen Allgemeinheiten und das Naheliegendste auch Bekanntes/Anekdoten Stereotypen das Beste ist Bestätigungen vereinfachten wir beenden sollten, was Wahrscheinlichkeiten und wir angefangen haben Zahlen unsere Optionen einfacheren -
Reproducibility: Promoting Scientific Rigor and Transparency
Reproducibility: Promoting scientific rigor and transparency Roma Konecky, PhD Editorial Quality Advisor What does reproducibility mean? • Reproducibility is the ability to generate similar results each time an experiment is duplicated. • Data reproducibility enables us to validate experimental results. • Reproducibility is a key part of the scientific process; however, many scientific findings are not replicable. The Reproducibility Crisis • ~2010 as part of a growing awareness that many scientific studies are not replicable, the phrase “Reproducibility Crisis” was coined. • An initiative of the Center for Open Science conducted replications of 100 psychology experiments published in prominent journals. (Science, 349 (6251), 28 Aug 2015) - Out of 100 replication attempts, only 39 were successful. The Reproducibility Crisis • According to a poll of over 1,500 scientists, 70% had failed to reproduce at least one other scientist's experiment or their own. (Nature 533 (437), 26 May 2016) • Irreproducible research is a major concern because in valid claims: - slow scientific progress - waste time and resources - contribute to the public’s mistrust of science Factors contributing to Over 80% of respondents irreproducibility Nature | News Feature 25 May 2016 Underspecified methods Factors Data dredging/ Low statistical contributing to p-hacking power irreproducibility Technical Bias - omitting errors null results Weak experimental design Underspecified methods Factors Data dredging/ Low statistical contributing to p-hacking power irreproducibility Technical Bias - omitting errors null results Weak experimental design Underspecified methods • When experimental details are omitted, the procedure needed to reproduce a study isn’t clear. • Underspecified methods are like providing only part of a recipe. ? = Underspecified methods Underspecified methods Underspecified methods Underspecified methods • Like baking a loaf of bread, a “scientific recipe” should include all the details needed to reproduce the study. -
Rejecting Statistical Significance Tests: Defanging the Arguments
Rejecting Statistical Significance Tests: Defanging the Arguments D. G. Mayo Philosophy Department, Virginia Tech, 235 Major Williams Hall, Blacksburg VA 24060 Abstract I critically analyze three groups of arguments for rejecting statistical significance tests (don’t say ‘significance’, don’t use P-value thresholds), as espoused in the 2019 Editorial of The American Statistician (Wasserstein, Schirm and Lazar 2019). The strongest argument supposes that banning P-value thresholds would diminish P-hacking and data dredging. I argue that it is the opposite. In a world without thresholds, it would be harder to hold accountable those who fail to meet a predesignated threshold by dint of data dredging. Forgoing predesignated thresholds obstructs error control. If an account cannot say about any outcomes that they will not count as evidence for a claim—if all thresholds are abandoned—then there is no a test of that claim. Giving up on tests means forgoing statistical falsification. The second group of arguments constitutes a series of strawperson fallacies in which statistical significance tests are too readily identified with classic abuses of tests. The logical principle of charity is violated. The third group rests on implicit arguments. The first in this group presupposes, without argument, a different philosophy of statistics from the one underlying statistical significance tests; the second group—appeals to popularity and fear—only exacerbate the ‘perverse’ incentives underlying today’s replication crisis. Key Words: Fisher, Neyman and Pearson, replication crisis, statistical significance tests, strawperson fallacy, psychological appeals, 2016 ASA Statement on P-values 1. Introduction and Background Today’s crisis of replication gives a new urgency to critically appraising proposed statistical reforms intended to ameliorate the situation. -
Statistical Tips for Interpreting Scientific Claims
Research Skills Seminar Series 2019 CAHS Research Education Program Statistical Tips for Interpreting Scientific Claims Mark Jones Statistician, Telethon Kids Institute 18 October 2019 Research Skills Seminar Series | CAHS Research Education Program Department of Child Health Research | Child and Adolescent Health Service ResearchEducationProgram.org © CAHS Research Education Program, Department of Child Health Research, Child and Adolescent Health Service, WA 2019 Copyright to this material produced by the CAHS Research Education Program, Department of Child Health Research, Child and Adolescent Health Service, Western Australia, under the provisions of the Copyright Act 1968 (C’wth Australia). Apart from any fair dealing for personal, academic, research or non-commercial use, no part may be reproduced without written permission. The Department of Child Health Research is under no obligation to grant this permission. Please acknowledge the CAHS Research Education Program, Department of Child Health Research, Child and Adolescent Health Service when reproducing or quoting material from this source. Statistical Tips for Interpreting Scientific Claims CONTENTS: 1 PRESENTATION ............................................................................................................................... 1 2 ARTICLE: TWENTY TIPS FOR INTERPRETING SCIENTIFIC CLAIMS, SUTHERLAND, SPIEGELHALTER & BURGMAN, 2013 .................................................................................................................................. 15 3 -
Who Uses Base Rates and <Emphasis Type="Italic">P </Emphasis>(D
Memory & Cognition 1998,26 (1), 161-179 Who uses base rates and P(D/,..,H)? An analysis ofindividual differences KEITHE. STANOVICH University ofToronto, Toronto, Ontario, Canada and RICHARD F. WEST James Madison University, Harrisonburg, Virginia In two experiments, involving over 900 subjects, we examined the cognitive correlates of the ten dency to view P(D/- H) and base rate information as relevant to probability assessment. Wefound that individuals who viewed P(D/- H) as relevant in a selection task and who used it to make the proper Bayesian adjustment in a probability assessment task scored higher on tests of cognitive ability and were better deductive and inductive reasoners. They were less biased by prior beliefs and more data driven on a covariation assessment task. In contrast, individuals who thought that base rates were rel evant did not display better reasoning skill or higher cognitive ability. Our results parallel disputes about the normative status of various components of the Bayesian formula in interesting ways. Itis ar gued that patterns of covariance among reasoning tasks may have implications for inferences about what individuals are trying to optimize in a rational analysis (J. R. Anderson, 1990, 1991). Twodeviations from normatively correct Bayesian rea and were asked to choose which pieces of information soning have been the focus ofmuch research. The two de they would need in order to determine whether the patient viations are most easily characterized ifBayes' rule is ex had the disease "Digirosa," The four pieces of informa pressed in the ratio form, where the odds favoring the focal tion were the percentage ofpeople with Digirosa, the per hypothesis (H) are derived by multiplying the likelihood centage of people without Digirosa, the percentage of ratio ofthe observed datum (D) by the prior odds favor people with Digirosa who have a red rash, and the per ing the focal hypothesis: centage ofpeople without Digirosa who have a red rash. -
Is Racial Profiling a Legitimate Strategy in the Fight Against Violent Crime?
Philosophia https://doi.org/10.1007/s11406-018-9945-1 Is Racial Profiling a Legitimate Strategy in the Fight against Violent Crime? Neven Sesardić1 Received: 21 August 2017 /Revised: 15 December 2017 /Accepted: 2 January 2018 # Springer Science+Business Media B.V., part of Springer Nature 2018 Abstract Racial profiling has come under intense public scrutiny especially since the rise of the Black Lives Matter movement. This article discusses two questions: (1) whether racial profiling is sometimes rational, and (2) whether it can be morally permissible. It is argued that under certain circumstances the affirmative answer to both questions is justified. Keywords Racial profiling . Discrimination . Police racism . Black lives matter. Bayes’s theorem . Base rate fallacy. Group differences 1 Introduction The Black Lives Matter (BLM) movement is driven by the belief that the police systematically discriminate against blacks. If such discrimination really occurs, is it necessarily morally unjustified? The question sounds like a provocation. For isn’t it obviously wrong to treat people differently just because they differ with respect to an inconsequential attribute like race or skin color? Indeed, racial profiling does go against Martin Luther King’s dream of a color-blind society where people Bwill not be judged by the color of their skin, but by the content of their character.^ The key question, however, is whether this ideal is achievable in the real world, as it is today, with some persistent statistical differences between the groups. Figure 1 shows group differences in homicide rates over a 29-year period, according to the Department of Justice (DOJ 2011:3): As we see (the red rectangle), the black/white ratio of the frequency of homicide offenders in these two groups is 7.6 (i.e., 34.4/4.5). -
Drøfting Og Argumentasjon
Bibliotekarstudentens nettleksikon om litteratur og medier Av Helge Ridderstrøm (førsteamanuensis ved OsloMet – storbyuniversitetet) Sist oppdatert 04.12.20 Drøfting og argumentasjon Ordet drøfting er norsk, og innebærer en diskuterende og problematiserende saksframstilling, med argumentasjon. Det er en systematisk diskusjon (ofte med seg selv). Å drøfte betyr å finne argumenter for og imot – som kan støtte eller svekke en påstand. Drøfting er en type kritisk tenking og refleksjon. Det kan beskrives som “systematisk og kritisk refleksjon” (Stene 2003 s. 106). Ordet brukes også som sjangerbetegnelse. Drøfting krever selvstendig tenkning der noe vurderes fra ulike ståsteder/ perspektiver. Den som drøfter, må vise evne til flersidighet. Et emne eller en sak ses fra flere sider, det veies for og imot noe, med styrker og svakheter, fordeler og ulemper. Den personen som drøfter, er på en måte mange personer, perspektiver og innfallsvinkler i én og samme person – først i sitt eget hode, og deretter i sin tekst. Idealet har blitt kalt “multiperspektivering”, dvs. å se noe fra mange sider (Nielsen 2010 s. 108). “Å drøfte er å diskutere med deg selv. En drøfting består av argumentasjon hvor du undersøker og diskuterer et fenomen fra flere sider. Drøfting er å kommentere og stille spørsmål til det du har redegjort for. Å drøfte er ikke bare å liste opp hva ulike forfattere sier, men å vise at det finnes ulike tolkninger og forståelser av hva de skriver om. Gjennom å vurdere de ulike tolkningene opp mot problemstillingen, foretar du en drøfting.” (https://student.hioa.no/argumentasjon-drofting-refleksjon; lesedato 19.10.18) I en drøfting inngår det å “vurdere fagkunnskap fra ulike perspektiver og ulike sider.