Natural Experiments and Instrumental Variables Analyses in Social Epidemiology
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Introduction to Biostatistics
Introduction to Biostatistics Jie Yang, Ph.D. Associate Professor Department of Family, Population and Preventive Medicine Director Biostatistical Consulting Core Director Biostatistics and Bioinformatics Shared Resource, Stony Brook Cancer Center In collaboration with Clinical Translational Science Center (CTSC) and the Biostatistics and Bioinformatics Shared Resource (BB-SR), Stony Brook Cancer Center (SBCC). OUTLINE What is Biostatistics What does a biostatistician do • Experiment design, clinical trial design • Descriptive and Inferential analysis • Result interpretation What you should bring while consulting with a biostatistician WHAT IS BIOSTATISTICS • The science of (bio)statistics encompasses the design of biological/clinical experiments the collection, summarization, and analysis of data from those experiments the interpretation of, and inference from, the results How to Lie with Statistics (1954) by Darrell Huff. http://www.youtube.com/watch?v=PbODigCZqL8 GOAL OF STATISTICS Sampling POPULATION Probability SAMPLE Theory Descriptive Descriptive Statistics Statistics Inference Population Sample Parameters: Inferential Statistics Statistics: 흁, 흈, 흅… 푿ഥ , 풔, 풑ෝ,… PROPERTIES OF A “GOOD” SAMPLE • Adequate sample size (statistical power) • Random selection (representative) Sampling Techniques: 1.Simple random sampling 2.Stratified sampling 3.Systematic sampling 4.Cluster sampling 5.Convenience sampling STUDY DESIGN EXPERIEMENT DESIGN Completely Randomized Design (CRD) - Randomly assign the experiment units to the treatments -
Experimentation Science: a Process Approach for the Complete Design of an Experiment
Kansas State University Libraries New Prairie Press Conference on Applied Statistics in Agriculture 1996 - 8th Annual Conference Proceedings EXPERIMENTATION SCIENCE: A PROCESS APPROACH FOR THE COMPLETE DESIGN OF AN EXPERIMENT D. D. Kratzer K. A. Ash Follow this and additional works at: https://newprairiepress.org/agstatconference Part of the Agriculture Commons, and the Applied Statistics Commons This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License. Recommended Citation Kratzer, D. D. and Ash, K. A. (1996). "EXPERIMENTATION SCIENCE: A PROCESS APPROACH FOR THE COMPLETE DESIGN OF AN EXPERIMENT," Conference on Applied Statistics in Agriculture. https://doi.org/ 10.4148/2475-7772.1322 This is brought to you for free and open access by the Conferences at New Prairie Press. It has been accepted for inclusion in Conference on Applied Statistics in Agriculture by an authorized administrator of New Prairie Press. For more information, please contact [email protected]. Conference on Applied Statistics in Agriculture Kansas State University Applied Statistics in Agriculture 109 EXPERIMENTATION SCIENCE: A PROCESS APPROACH FOR THE COMPLETE DESIGN OF AN EXPERIMENT. D. D. Kratzer Ph.D., Pharmacia and Upjohn Inc., Kalamazoo MI, and K. A. Ash D.V.M., Ph.D., Town and Country Animal Hospital, Charlotte MI ABSTRACT Experimentation Science is introduced as a process through which the necessary steps of experimental design are all sufficiently addressed. Experimentation Science is defined as a nearly linear process of objective formulation, selection of experimentation unit and decision variable(s), deciding treatment, design and error structure, defining the randomization, statistical analyses and decision procedures, outlining quality control procedures for data collection, and finally analysis, presentation and interpretation of results. -
Natural Experiments
Natural Experiments Jason Seawright [email protected] August 11, 2010 J. Seawright (PolSci) Essex 2010 August 11, 2010 1 / 31 Typology of Natural Experiments J. Seawright (PolSci) Essex 2010 August 11, 2010 2 / 31 Typology of Natural Experiments Classic Natural Experiment J. Seawright (PolSci) Essex 2010 August 11, 2010 2 / 31 Typology of Natural Experiments Classic Natural Experiment Instrumental Variables-Type Natural Experiment J. Seawright (PolSci) Essex 2010 August 11, 2010 2 / 31 Typology of Natural Experiments Classic Natural Experiment Instrumental Variables-Type Natural Experiment Regression-Discontinuity Design J. Seawright (PolSci) Essex 2010 August 11, 2010 2 / 31 Bolstering Understand assumptions. J. Seawright (PolSci) Essex 2010 August 11, 2010 3 / 31 Bolstering Understand assumptions. Explore role of qualitative evidence. J. Seawright (PolSci) Essex 2010 August 11, 2010 3 / 31 Classic Natural Experiment J. Seawright (PolSci) Essex 2010 August 11, 2010 4 / 31 Classic Natural Experiment 1 “Nature” randomizes the treatment. J. Seawright (PolSci) Essex 2010 August 11, 2010 4 / 31 Classic Natural Experiment 1 “Nature” randomizes the treatment. 2 All (observable and unobservable) confounding variables are balanced between treatment and control groups. J. Seawright (PolSci) Essex 2010 August 11, 2010 4 / 31 Classic Natural Experiment 1 “Nature” randomizes the treatment. 2 All (observable and unobservable) confounding variables are balanced between treatment and control groups. 3 No discretion is involved in assigning treatments, or the relevant information is unavailable or unused. J. Seawright (PolSci) Essex 2010 August 11, 2010 4 / 31 Classic Natural Experiment 1 “Nature” randomizes the treatment. 2 All (observable and unobservable) confounding variables are balanced between treatment and control groups. -
Toward an Ethical Experiment
TOWARD AN ETHICAL EXPERIMENT By Yusuke Narita April 2018 COWLES FOUNDATION DISCUSSION PAPER NO. 2127 COWLES FOUNDATION FOR RESEARCH IN ECONOMICS YALE UNIVERSITY Box 208281 New Haven, Connecticut 06520-8281 http://cowles.yale.edu/ ∗ Toward an Ethical Experiment y Yusuke Narita April 17, 2018 Abstract Randomized Controlled Trials (RCTs) enroll hundreds of millions of subjects and in- volve many human lives. To improve subjects' welfare, I propose an alternative design of RCTs that I call Experiment-as-Market (EXAM). EXAM Pareto optimally randomly assigns each treatment to subjects predicted to experience better treatment effects or to subjects with stronger preferences for the treatment. EXAM is also asymptotically incentive compatible for preference elicitation. Finally, EXAM unbiasedly estimates any causal effect estimable with standard RCTs. I quantify the welfare, incentive, and information properties by applying EXAM to a water cleaning experiment in Kenya (Kremer et al., 2011). Compared to standard RCTs, EXAM substantially improves subjects' predicted well-being while reaching similar treatment effect estimates with similar precision. Keywords: Research Ethics, Clinical Trial, Social Experiment, A/B Test, Market De- sign, Causal Inference, Development Economics, Spring Protection, Discrete Choice ∗I am grateful to Dean Karlan for a conversation that inspired this project; Joseph Moon for industrial and institutional input; Jason Abaluck, Josh Angrist, Tim Armstrong, Sylvain Chassang, Naoki Egami, Peter Hull, Costas Meghir, Bobby Pakzad-Hurson, -
The Distribution of Information in Speculative Markets: a Natural Experiment∗
The Distribution of Information in Speculative Markets: A Natural Experiment∗ Alasdair Brown† University of East Anglia June 9, 2014 ∗I would like to thank an associate editor, two anonymous referees, Fuyu Yang, and seminar participants at UEA for comments on this work. All remaining errors are my own. †School of Economics, University of East Anglia, Norwich, NR4 7TJ. Email: [email protected] 1 1 Introduction Abstract We use a unique natural experiment to shed light on the distribution of information in speculative markets. In June 2011, Betfair - a U.K. betting exchange - levied a tax of up to 60% on all future profits accrued by the top 0.1% of profitable traders. Such a move appears to have driven at least some of these traders off the exchange, taking their information with them. We investigate the effect of the new tax on the forecasting capacity of the exchange (our measure of the market’s incorporation of information into the price). We find that there was scant decline in the forecasting capacity of the exchange - relative to a control market - suggesting that the bulk of information had hitherto been held by the majority of traders, rather than the select few affected by the rule change. This result is robust to the choice of forecasting measure, the choice of forecasting interval, and the choice of race type. This provides evidence that more than a few traders are typically involved in the price discovery process in speculative markets. JEL Classification: G14, G19 Keywords: informed trading, natural experiment, betting markets, horse racing 1 Introduction There is a common perception that the benefits of trading in speculative assets are skewed towards those on the inside of corporations, or those that move in the same circles as the corporate elite. -
Using Randomized Evaluations to Improve the Efficiency of US Healthcare Delivery
Using Randomized Evaluations to Improve the Efficiency of US Healthcare Delivery Amy Finkelstein Sarah Taubman MIT, J-PAL North America, and NBER MIT, J-PAL North America February 2015 Abstract: Randomized evaluations of interventions that may improve the efficiency of US healthcare delivery are unfortunately rare. Across top journals in medicine, health services research, and economics, less than 20 percent of studies of interventions in US healthcare delivery are randomized. By contrast, about 80 percent of studies of medical interventions are randomized. The tide may be turning toward making randomized evaluations closer to the norm in healthcare delivery, as the relevant actors have an increasing need for rigorous evidence of what interventions work and why. At the same time, the increasing availability of administrative data that enables high-quality, low-cost RCTs and of new sources of funding that support RCTs in healthcare delivery make them easier to conduct. We suggest a number of design choices that can enhance the feasibility and impact of RCTs on US healthcare delivery including: greater reliance on existing administrative data; measuring a wide range of outcomes on healthcare costs, health, and broader economic measures; and designing experiments in a way that sheds light on underlying mechanisms. Finally, we discuss some of the more common concerns raised about the feasibility and desirability of healthcare RCTs and when and how these can be overcome. _____________________________ We are grateful to Innessa Colaiacovo, Lizi Chen, and Belinda Tang for outstanding research assistance, to Katherine Baicker, Mary Ann Bates, Kelly Bidwell, Joe Doyle, Mireille Jacobson, Larry Katz, Adam Sacarny, Jesse Shapiro, and Annetta Zhou for helpful comments, and to the Laura and John Arnold Foundation for financial support. -
Double Blind Trials Workshop
Double Blind Trials Workshop Introduction These activities demonstrate how double blind trials are run, explaining what a placebo is and how the placebo effect works, how bias is removed as far as possible and how participants and trial medicines are randomised. Curriculum Links KS3: Science SQA Access, Intermediate and KS4: Biology Higher: Biology Keywords Double-blind trials randomisation observer bias clinical trials placebo effect designing a fair trial placebo Contents Activities Materials Activity 1 Placebo Effect Activity Activity 2 Observer Bias Activity 3 Double Blind Trial Role Cards for the Double Blind Trial Activity Testing Layout Background Information Medicines undergo a number of trials before they are declared fit for use (see classroom activity on Clinical Research for details). In the trial in the second activity, pupils compare two potential new sunscreens. This type of trial is done with healthy volunteers to see if the there are any side effects and to provide data to suggest the dosage needed. If there were no current best treatment then this sort of trial would also be done with patients to test for the effectiveness of the new medicine. How do scientists make sure that medicines are tested fairly? One thing they need to do is to find out if their tests are free of bias. Are the medicines really working, or do they just appear to be working? One difficulty in designing fair tests for medicines is the placebo effect. When patients are prescribed a treatment, especially by a doctor or expert they trust, the patient’s own belief in the treatment can cause the patient to produce a response. -
Observational Studies and Bias in Epidemiology
The Young Epidemiology Scholars Program (YES) is supported by The Robert Wood Johnson Foundation and administered by the College Board. Observational Studies and Bias in Epidemiology Manuel Bayona Department of Epidemiology School of Public Health University of North Texas Fort Worth, Texas and Chris Olsen Mathematics Department George Washington High School Cedar Rapids, Iowa Observational Studies and Bias in Epidemiology Contents Lesson Plan . 3 The Logic of Inference in Science . 8 The Logic of Observational Studies and the Problem of Bias . 15 Characteristics of the Relative Risk When Random Sampling . and Not . 19 Types of Bias . 20 Selection Bias . 21 Information Bias . 23 Conclusion . 24 Take-Home, Open-Book Quiz (Student Version) . 25 Take-Home, Open-Book Quiz (Teacher’s Answer Key) . 27 In-Class Exercise (Student Version) . 30 In-Class Exercise (Teacher’s Answer Key) . 32 Bias in Epidemiologic Research (Examination) (Student Version) . 33 Bias in Epidemiologic Research (Examination with Answers) (Teacher’s Answer Key) . 35 Copyright © 2004 by College Entrance Examination Board. All rights reserved. College Board, SAT and the acorn logo are registered trademarks of the College Entrance Examination Board. Other products and services may be trademarks of their respective owners. Visit College Board on the Web: www.collegeboard.com. Copyright © 2004. All rights reserved. 2 Observational Studies and Bias in Epidemiology Lesson Plan TITLE: Observational Studies and Bias in Epidemiology SUBJECT AREA: Biology, mathematics, statistics, environmental and health sciences GOAL: To identify and appreciate the effects of bias in epidemiologic research OBJECTIVES: 1. Introduce students to the principles and methods for interpreting the results of epidemio- logic research and bias 2. -
Social Psychology
Social Psychology OUTLINE OF RESOURCES Introducing Social Psychology Lecture/Discussion Topic: Social Psychology’s Most Important Lessons (p. 853) Social Thinking The Fundamental Attribution Error Lecture/Discussion Topic: Attribution and Models of Helping (p. 856) Classroom Exercises: The Fundamental Attribution Error (p. 854) Students’ Perceptions of You (p. 855) Classroom Exercise/Critical Thinking Break: Biases in Explaining Events (p. 855) NEW Feature (Short) Film: The Lunch Date (p. 856) Worth Video Anthology: The Actor-Observer Difference in Attribution: Observe a Riot in Action* Attitudes and Actions Lecture/Discussion Topics: The Looking Glass Effect (p. 856) The Theory of Reasoned Action (p. 857) Actions Influence Attitudes (p. 857) The Justification of Effort (p. 858) Self-Persuasion (p. 859) Revisiting the Stanford Prison Experiment (p. 859) Abu Ghraib Prison and Social Psychology (p. 860) Classroom Exercise: Introducing Cognitive Dissonance Theory (p. 858) Worth Video Anthology: Zimbardo’s Stanford Prison Experiment* The Stanford Prison Experiment: The Power of the Situation* Social Influence Conformity: Complying With Social Pressures Lecture/Discussion Topics: Mimicry and Prosocial Behavior (p. 861) Social Exclusion and Mimicry (p. 861) The Seattle Windshield Pitting Epidemic (p. 862) Classroom Exercises: Suggestibility (p. 862) Social Influence (p. 863) Student Project: Violating a Social Norm (p. 863) Worth Video Anthology: Social Influence* NEW Liking and Imitation: The Sincerest Form of Flattery* Obedience: Following Orders Lecture/Discussion Topic: Obedience in Everyday Life (p. 865) Classroom Exercises: Obedience and Conformity (p. 864) Would You Obey? (p. 864) Wolves or Sheep? (p. 866) * Titles in the Worth Video Anthology are not described within the core resource unit. -
Design of Experiments and Data Analysis,” 2012 Reliability and Maintainability Symposium, January, 2012
Copyright © 2012 IEEE. Reprinted, with permission, from Huairui Guo and Adamantios Mettas, “Design of Experiments and Data Analysis,” 2012 Reliability and Maintainability Symposium, January, 2012. This material is posted here with permission of the IEEE. Such permission of the IEEE does not in any way imply IEEE endorsement of any of ReliaSoft Corporation's products or services. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing to [email protected]. By choosing to view this document, you agree to all provisions of the copyright laws protecting it. 2012 Annual RELIABILITY and MAINTAINABILITY Symposium Design of Experiments and Data Analysis Huairui Guo, Ph. D. & Adamantios Mettas Huairui Guo, Ph.D., CPR. Adamantios Mettas, CPR ReliaSoft Corporation ReliaSoft Corporation 1450 S. Eastside Loop 1450 S. Eastside Loop Tucson, AZ 85710 USA Tucson, AZ 85710 USA e-mail: [email protected] e-mail: [email protected] Tutorial Notes © 2012 AR&MS SUMMARY & PURPOSE Design of Experiments (DOE) is one of the most useful statistical tools in product design and testing. While many organizations benefit from designed experiments, others are getting data with little useful information and wasting resources because of experiments that have not been carefully designed. Design of Experiments can be applied in many areas including but not limited to: design comparisons, variable identification, design optimization, process control and product performance prediction. Different design types in DOE have been developed for different purposes. -
A Randomized Control Trial Evaluating the Effects of Police Body-Worn
A randomized control trial evaluating the effects of police body-worn cameras David Yokuma,b,1,2, Anita Ravishankara,c,d,1, and Alexander Coppocke,1 aThe Lab @ DC, Office of the City Administrator, Executive Office of the Mayor, Washington, DC 20004; bThe Policy Lab, Brown University, Providence, RI 02912; cExecutive Office of the Chief of Police, Metropolitan Police Department, Washington, DC 20024; dPublic Policy and Political Science Joint PhD Program, University of Michigan, Ann Arbor, MI 48109; and eDepartment of Political Science, Yale University, New Haven, CT 06511 Edited by Susan A. Murphy, Harvard University, Cambridge, MA, and approved March 21, 2019 (received for review August 28, 2018) Police body-worn cameras (BWCs) have been widely promoted as The existing evidence on whether BWCs have the anticipated a technological mechanism to improve policing and the perceived effects on policing outcomes remains relatively limited (17–19). legitimacy of police and legal institutions, yet evidence of their Several observational studies have evaluated BWCs by compar- effectiveness is limited. To estimate the effects of BWCs, we con- ing the behavior of officers before and after the introduction of ducted a randomized controlled trial involving 2,224 Metropolitan BWCs into the police department (20, 21). Other studies com- Police Department officers in Washington, DC. Here we show pared officers who happened to wear BWCs to those without (15, that BWCs have very small and statistically insignificant effects 22, 23). The causal inferences drawn in those studies depend on on police use of force and civilian complaints, as well as other strong assumptions about whether, after statistical adjustments policing activities and judicial outcomes. -
Synthetic Control Method: Inference, Sensitivity Analysis and Confidence Sets∗
Synthetic Control Method: Inference, Sensitivity Analysis and Confidence Sets∗ Sergio Firpo† Vitor Possebom‡ Insper Yale University September 1st, 2017 Abstract We extend the inference procedure for the synthetic control method in two ways. First, we impose a parametric form to the treatment assignment probabilities that includes, as a particular case, the uniform distribution that is assumed by Abadie et al.(2015). By changing the value of this parameter, we can analyze the sensitiveness of the test's decision to deviations from the usual uniform distribution assumption. Second, we modify the RMSPE statistic to test any sharp null hypothesis, including, as a specific case, the null hypothesis of no effect whatsoever analyzed by Abadie et al.(2015). Based on this last extension, we invert the test statistic to estimate confidence sets that quickly show the point-estimates' precision, and the test's significance and robustness. We also extend these two tools to other test statistics and to problems with multiple outcome variables or multiple treated units. Furthermore, in a Monte Carlo experiment, we find that the RMSPE statistic has good properties with respect to size, power and robustness. Finally, we illustrate the usefulness of our proposed tools by reanalyzing the economic impact of ETA's terrorism in the Basque Country, studied first by Abadie & Gardeazabal(2003) and Abadie et al.(2011). Keywords: Synthetic Control Estimator, Hypothesis Testing, Sensitivity Analysis, Con- fidence Sets JEL Codes: C21, C23, C33 ∗We are grateful to FAPESP that provided financial aid through grant number 2014/23731-3. We are in debt to the editor and two anonymous referees for useful suggestions.