DOCSLIB.ORG

Introduction to Observational Studies and the Reprint of Cochran's Paper “Observational Studies” and Comments

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Introduction to Observational Studies and the Reprint of Cochran's Paper “Observational Studies” and Comments Observational Studies 1 (2015) 124-125 Submitted 8/15; Published 8/15 Introduction to Observational Studies and the Reprint of Cochran's paper \Observational Studies" and Comments Dylan S. Small [email protected] Department of Statistics, University of Pennsylvania Philadelphia, PA 19104, USA In this first issue of Observational Studies, we reprint a review of observational studies by William Cochran, a pioneer of statistical research on observational studies, followed by comments by leading current researchers in observational studies. Cochran (1965, Journal of the Royal Statistical Society, Series A) defined an observational study as an empiric investigation [in which]...the objective is to elucidate cause-and-effect relationships...[in which] it is not feasible to use controlled experimentation, in the sense of being able to impose the procedures or treatments whose effects it is desired to discover, or to assign subjects at random to different procedures. Observational Studies is a new peer-reviewed journal that seeks to publish papers on all aspects of observational studies. Researchers from all fields that make use of observational studies are encouraged to submit papers. Topics covered by the journal include, but are not limited, to the following: • Study protocols for observational studies. The journal seeks to promote the planning and transparency of observational studies. In addition to publishing study protocols, the journal will publish comments on the study protocols and allow the authors of the study protocol to respond to the comments. • Methodologies for observational studies. This includes statistical methods for all as- pects of observational studies and methods for the conduct of observational studies such as methods for collecting data. In addition to novel methodological articles, the journal welcomes review articles on methodology relevant to observational studies as well as illustrations/explanations of methodologies that may have been developed in a more technical article in another journal. • Software for observational studies. The journal welcomes articles describing software relevant to observational studies. • Descriptions of observational study data sets. The journal welcomes descriptions of observational study data sets and how to access them. The goal of the descriptions of observational study data sets is to enable readers to form collaborations, to learn from each other and to maximize use of existing resources. The journal also encourages submission of examples of how a publicly available observational study database can be used. ⃝c 2015 Dylan S. Small. • Analyses of observational studies. The journal welcomes analyses of observational studies. The journal encourages submissions of analyses that illustrate use of sound methodology and conduct of observational studies. The paper we reprint of Cochran's and the comments by leading current researchers in observational studies provide illuminating perspectives on important issues in observational studies that the journal seeks to address. The contents of the rest of this section are as follows: Author Title Pages William Cochran Observational Studies 126-136 Norman Breslow William G. Cochran and the 1964 Surgeon's 137-140 General Report Thomas Cook The Inheritance bequeathed to William G. 140-163 Cochran that he willed forward and left for others to will forward again: The Limits of Observational Studies that seek to Mimic Randomized Experiments David Cox & Nanny Wermuth Design and interpretation of studies: relevant 165{170 concepts from the past and some extensions Stephen Fienberg Comment on \Observational Studies" 171{172 by William G. Cochran Joseph Gastwirth & Barry Graubard Comment on Cochran's \Observational Studies 173{181 Andrew Gelman The State of the Art in Causal Inference: 182{183 Some Changes Since 1972 Ben Hansen & Adam Sales Comment on Cochran's \Observational Studies" 184{193 Miguel Hern´an A good deal of humility: 194{195 Cochran on observational studies Jennifer Hill Lessons we are still learning 196{199 Judea Pearl Causal Thinking in the Twilight Zone 200{204 Paul Rosenbaum Cochran's Causal Crossword 205{211 Donald Rubin Comment on Cochran's \Observational Studies" 212{216 Herbert Smith Comment on Cochran's \Observational Studies" 217{219 Mark van der Laan Comment on \Observational Studies" 220{222 by Dr. W.G. Cochran (1972) Tyler VanderWeele Observational Studies and Study Designs: 223{230 An Epidemiologic Perspective Stephen West Reflections on \Observational Studies": 231{240 Looking Backward and Looking Forward 125 Observational Studies 1 (2015) 126-136 Submitted 1972; Published reprinted, 8/15 Observational Studies William G. Cochran Editor's Note: William G. Cochran (1909-1980) was Professor of Statistics, Harvard University, Cambridge, Massachusetts. This article was originally published in Statistical Papers in Honor of George W. Snedecor, ed. T.A. Bancroft, 1972, Iowa State University Press, pp. 77-90. The paper is reprinted with permission of the copyright holder, Iowa State University Press. Comments by leading current researchers in observational studies follow. 1. Introduction OBSERVATIONAL STUDIES are a class of statistical studies that have increased in fre- quency and importance during the past 20 years. In an observational study the investigator is restricted to taking selected observations or measurements on the process under study. For one reason or another he cannot interfere in the process in the way that one does in a controlled laboratory type of experiment. Observational studies fall roughly into two broad types. The first is often given the name of \analytical surveys." The investigator takes a sample survey of a population of interest and proceeds to conduct statistical analyses of the relations between variables of interest to him. An early example was Kinsey's study (1948) of the relation between the frequencies of certain types of sexual behavior and variables like the age, sex, social level, religious affiliation, rural-urban background, and direction of social mobility of the person involved. Dr. Kinsey gave much thought to the methodological problems that he would face in planning his study. More recently, in what is called the \midtown Manhattan study" (Srole et al., 1962), a team of psychiatrists studied the relation in Manhattan, New York, between age, sex, parental and own social level, ethnic origin, generation in the United States, and religion and nonhospitalized mental illness. The second type of observational study is narrower in scope. The investigator has in mind some agents, procedures, or experiences that may produce certain causal effects (good or bad) on people. These agents are like those the statistician would call treatments in a controlled experiment, except that a controlled experiment is not feasible. Examples of this type abound. A simple one structurally is a Cornell study of the effect of wearing a lap seat belt on the amount and type of injury sustained in an automobile collision. This study was done from police and medical records of injuries in automobile accidents. The prospective smoking and health studies (1964) are also a well-known example. These are comparisons of the death rates and causes of death of men and women with different smoking patterns in regard to type and amount. An example known as the \national halothane study" (Bunker et al., 1969) attempted to make a fair comparison of the death rates due to the five leading anesthetics used in hospital operations. ⃝c 2015 Iowa State University Press. Observational Studies Several factors are probably responsible for the growth in the number of studies of this kind. One is a general increase in funds for research in the social sciences and medicine. A related reason is the growing awareness of social problems. A study known as the \Coleman report" (1966) has attracted much discussion. This was begun because Congress gave the U.S. Office of Education a substantial sum and asked it to conduct a nation wide survey of elementary schools and high schools to discover to what extent minority-group children in the United States (Blacks, Indians, Puerto Ricans, Mexican-Americans, and Orientals) receive a poorer education than the majority whites. A third reason is the growing area of program evaluation. All over the world, administrative bodies { central, regional, and local { spend the taxpayers' money on new programs intended to benefit some or all of the population or to combat social evils. Similarly, a business organization may institute changes in its operations in the hope of improving the running of the business. The idea is spreading that it might be wise to devote some resources to trying to measure both the intended and the unintended effects of these programs. Such evaluations are difficult to do well, and they make much use of observational studies. Finally, some studies are undertaken to investigate stray reports of unexpected effects that appear from time to time. The halothane study is an example; others are studies of side effects of the contraceptive pill and studies of health effects of air pollution. This paper is confined mainly to the second, narrower class of observational studies, although some of the problems to be considered are also met in the broader analytical ones. For this paper I naturally sought a topic that would reflect the outlook and research interests of George Snedecor. In his career activity of helping investigators, he developed a strong interest in the design of experiments, a subject on
Load more

Recommended publications
  • Natural Experiments
    Natural Experiments Jason Seawright [email protected] August 11, 2010 J. Seawright (PolSci) Essex 2010 August 11, 2010 1 / 31 Typology of Natural Experiments J. Seawright (PolSci) Essex 2010 August 11, 2010 2 / 31 Typology of Natural Experiments Classic Natural Experiment J. Seawright (PolSci) Essex 2010 August 11, 2010 2 / 31 Typology of Natural Experiments Classic Natural Experiment Instrumental Variables-Type Natural Experiment J. Seawright (PolSci) Essex 2010 August 11, 2010 2 / 31 Typology of Natural Experiments Classic Natural Experiment Instrumental Variables-Type Natural Experiment Regression-Discontinuity Design J. Seawright (PolSci) Essex 2010 August 11, 2010 2 / 31 Bolstering Understand assumptions. J. Seawright (PolSci) Essex 2010 August 11, 2010 3 / 31 Bolstering Understand assumptions. Explore role of qualitative evidence. J. Seawright (PolSci) Essex 2010 August 11, 2010 3 / 31 Classic Natural Experiment J. Seawright (PolSci) Essex 2010 August 11, 2010 4 / 31 Classic Natural Experiment 1 “Nature” randomizes the treatment. J. Seawright (PolSci) Essex 2010 August 11, 2010 4 / 31 Classic Natural Experiment 1 “Nature” randomizes the treatment. 2 All (observable and unobservable) confounding variables are balanced between treatment and control groups. J. Seawright (PolSci) Essex 2010 August 11, 2010 4 / 31 Classic Natural Experiment 1 “Nature” randomizes the treatment. 2 All (observable and unobservable) confounding variables are balanced between treatment and control groups. 3 No discretion is involved in assigning treatments, or the relevant information is unavailable or unused. J. Seawright (PolSci) Essex 2010 August 11, 2010 4 / 31 Classic Natural Experiment 1 “Nature” randomizes the treatment. 2 All (observable and unobservable) confounding variables are balanced between treatment and control groups.
    [Show full text]
  • JEROME P. REITER Department of Statistical Science, Duke University Box 90251, Durham, NC 27708 Phone: 919 668 5227
    JEROME P. REITER Department of Statistical Science, Duke University Box 90251, Durham, NC 27708 phone: 919 668 5227. email: [email protected]. September 26, 2021 EDUCATION Ph.D. in Statistics, Harvard University, 1999. A.M. in Statistics, Harvard University, 1996. B.S. in Mathematics, Duke University, 1992. DISSERTATION \Estimation in the Presence of Constraints that Prohibit Explicit Data Pooling." Advisor: Donald B. Rubin. POSITIONS Academic Appointments Professor of Statistical Science and Bass Fellow, Duke University, 2015 - present. Mrs. Alexander Hehmeyer Professor of Statistical Science, Duke University, 2013 - 2015. Mrs. Alexander Hehmeyer Associate Professor of Statistical Science, Duke University, 2010 - 2013. Associate Professor of Statistical Science, Duke University, 2009 - 2010. Assistant Professor of Statistical Science, Duke University, 2006 - 2008. Assistant Professor of the Practice of Statistics and Decision Sciences, Duke University, 2002 - 2006. Lecturer in Statistics, University of California at Santa Barbara, 2001 - 2002. Assistant Professor of Statistics, Williams College, 1999 - 2001. Other Appointments Chair, Department of Statistical Science, Duke University, 2019 - present. Associate Chair, Department of Statistical Science, Duke University, 2016 - 2019. Mathematical Statistician (part-time), U. S. Bureau of the Census, 2015 - present. Associate/Deputy Director of Information Initiative at Duke, Duke University, 2013 - 2019. Social Sciences Research Institute Data Services Core Director, Duke University, 2010 - 2013. Interim Director, Triangle Research Data Center, 2006. Senior Fellow, National Institute of Statistical Sciences, 2002 - 2005. 1 ACADEMIC HONORS Keynote talk, 11th Official Statistics and Methodology Symposium (Statistics Korea), 2021. Fellow of the Institute of Mathematical Statistics, 2020. Clifford C. Clogg Memorial Lecture, Pennsylvania State University, 2020 (postponed due to covid-19).
    [Show full text]
  • The Distribution of Information in Speculative Markets: a Natural Experiment∗
    The Distribution of Information in Speculative Markets: A Natural Experiment∗ Alasdair Brown† University of East Anglia June 9, 2014 ∗I would like to thank an associate editor, two anonymous referees, Fuyu Yang, and seminar participants at UEA for comments on this work. All remaining errors are my own. †School of Economics, University of East Anglia, Norwich, NR4 7TJ. Email: [email protected] 1 1 Introduction Abstract We use a unique natural experiment to shed light on the distribution of information in speculative markets. In June 2011, Betfair - a U.K. betting exchange - levied a tax of up to 60% on all future profits accrued by the top 0.1% of profitable traders. Such a move appears to have driven at least some of these traders off the exchange, taking their information with them. We investigate the effect of the new tax on the forecasting capacity of the exchange (our measure of the market’s incorporation of information into the price). We find that there was scant decline in the forecasting capacity of the exchange - relative to a control market - suggesting that the bulk of information had hitherto been held by the majority of traders, rather than the select few affected by the rule change. This result is robust to the choice of forecasting measure, the choice of forecasting interval, and the choice of race type. This provides evidence that more than a few traders are typically involved in the price discovery process in speculative markets. JEL Classification: G14, G19 Keywords: informed trading, natural experiment, betting markets, horse racing 1 Introduction There is a common perception that the benefits of trading in speculative assets are skewed towards those on the inside of corporations, or those that move in the same circles as the corporate elite.
    [Show full text]
  • Observational Clinical Research
    E REVIEW ARTICLE Clinical Research Methodology 2: Observational Clinical Research Daniel I. Sessler, MD, and Peter B. Imrey, PhD * † Case-control and cohort studies are invaluable research tools and provide the strongest fea- sible research designs for addressing some questions. Case-control studies usually involve retrospective data collection. Cohort studies can involve retrospective, ambidirectional, or prospective data collection. Observational studies are subject to errors attributable to selec- tion bias, confounding, measurement bias, and reverse causation—in addition to errors of chance. Confounding can be statistically controlled to the extent that potential factors are known and accurately measured, but, in practice, bias and unknown confounders usually remain additional potential sources of error, often of unknown magnitude and clinical impact. Causality—the most clinically useful relation between exposure and outcome—can rarely be defnitively determined from observational studies because intentional, controlled manipu- lations of exposures are not involved. In this article, we review several types of observa- tional clinical research: case series, comparative case-control and cohort studies, and hybrid designs in which case-control analyses are performed on selected members of cohorts. We also discuss the analytic issues that arise when groups to be compared in an observational study, such as patients receiving different therapies, are not comparable in other respects. (Anesth Analg 2015;121:1043–51) bservational clinical studies are attractive because Group, and the American Society of Anesthesiologists they are relatively inexpensive and, perhaps more Anesthesia Quality Institute. importantly, can be performed quickly if the required Recent retrospective perioperative studies include data O 1,2 data are already available.
    [Show full text]
  • You May Be (Stuck) Here! and Here Are Some Potential Reasons Why
    You may be (stuck) here! And here are some potential reasons why. As published in Benchmarks RSS Matters, May 2015 http://web3.unt.edu/benchmarks/issues/2015/05/rss-matters Jon Starkweather, PhD 1 Jon Starkweather, PhD [email protected] Consultant Research and Statistical Support http://www.unt.edu http://www.unt.edu/rss RSS hosts a number of “Short Courses”. A list of them is available at: http://www.unt.edu/rss/Instructional.htm Those interested in learning more about R, or how to use it, can find information here: http://www.unt.edu/rss/class/Jon/R_SC 2 You may be (stuck) here! And here are some potential reasons why. I often read R-bloggers (Galili, 2015) to see new and exciting things users are doing in the wonderful world of R. Recently I came across Norm Matloff’s (2014) blog post with the title “Why are we still teaching t-tests?” To be honest, many RSS personnel have echoed Norm’s sentiments over the years. There do seem to be some fields which are perpetually stuck in decades long past — in terms of the statistical methods they teach and use. Reading Norm’s post got me thinking it might be good to offer some explanations, or at least opinions, on why some fields tend to be stubbornly behind the analytic times. This month’s article will offer some of my own thoughts on the matter. I offer these opinions having been academically raised in one such Rip Van Winkle (Washington, 1819) field and subsequently realized how much of what I was taught has very little practical utility with real world research problems and data.
    [Show full text]
  • Visualizing Research Collaboration in Statistical Science: a Scientometric
    University of Nebraska - Lincoln DigitalCommons@University of Nebraska - Lincoln Library Philosophy and Practice (e-journal) Libraries at University of Nebraska-Lincoln 9-13-2019 Visualizing Research Collaboration in Statistical Science: A Scientometric Perspective Prabir Kumar Das Library, Documentation & Information Science Division, Indian Statistical Institute, 203 B T Road, Kolkata - 700108, [email protected] Follow this and additional works at: https://digitalcommons.unl.edu/libphilprac Part of the Library and Information Science Commons Das, Prabir Kumar, "Visualizing Research Collaboration in Statistical Science: A Scientometric Perspective" (2019). Library Philosophy and Practice (e-journal). 3039. https://digitalcommons.unl.edu/libphilprac/3039 Viisualliiziing Research Collllaborattiion iin Sttattiisttiicall Sciience:: A Sciienttomettriic Perspecttiive Prrabiirr Kumarr Das Sciienttiiffiic Assssiissttantt – A Liibrrarry,, Documenttattiion & IInfforrmattiion Sciience Diiviisiion,, IIndiian Sttattiisttiicall IInsttiittutte,, 203,, B.. T.. Road,, Kollkatta – 700108,, IIndiia,, Emaiill:: [email protected] Abstract Using Sankhyā – The Indian Journal of Statistics as a case, present study aims to identify scholarly collaboration pattern of statistical science based on research articles appeared during 2008 to 2017. This is an attempt to visualize and quantify statistical science research collaboration in multiple dimensions by exploring the co-authorship data. It investigates chronological variations of collaboration pattern, nodes and links established among the affiliated institutions and countries of all contributing authors. The study also examines the impact of research collaboration on citation scores. Findings reveal steady influx of statistical publications with clear tendency towards collaborative ventures, of which double-authored publications dominate. Small team of 2 to 3 authors is responsible for production of majority of collaborative research, whereas mega-authored communications are quite low.
    [Show full text]
  • Observational Studies and Bias in Epidemiology
    The Young Epidemiology Scholars Program (YES) is supported by The Robert Wood Johnson Foundation and administered by the College Board. Observational Studies and Bias in Epidemiology Manuel Bayona Department of Epidemiology School of Public Health University of North Texas Fort Worth, Texas and Chris Olsen Mathematics Department George Washington High School Cedar Rapids, Iowa Observational Studies and Bias in Epidemiology Contents Lesson Plan . 3 The Logic of Inference in Science . 8 The Logic of Observational Studies and the Problem of Bias . 15 Characteristics of the Relative Risk When Random Sampling . and Not . 19 Types of Bias . 20 Selection Bias . 21 Information Bias . 23 Conclusion . 24 Take-Home, Open-Book Quiz (Student Version) . 25 Take-Home, Open-Book Quiz (Teacher’s Answer Key) . 27 In-Class Exercise (Student Version) . 30 In-Class Exercise (Teacher’s Answer Key) . 32 Bias in Epidemiologic Research (Examination) (Student Version) . 33 Bias in Epidemiologic Research (Examination with Answers) (Teacher’s Answer Key) . 35 Copyright © 2004 by College Entrance Examination Board. All rights reserved. College Board, SAT and the acorn logo are registered trademarks of the College Entrance Examination Board. Other products and services may be trademarks of their respective owners. Visit College Board on the Web: www.collegeboard.com. Copyright © 2004. All rights reserved. 2 Observational Studies and Bias in Epidemiology Lesson Plan TITLE: Observational Studies and Bias in Epidemiology SUBJECT AREA: Biology, mathematics, statistics, environmental and health sciences GOAL: To identify and appreciate the effects of bias in epidemiologic research OBJECTIVES: 1. Introduce students to the principles and methods for interpreting the results of epidemio- logic research and bias 2.
    [Show full text]
  • Study Types Transcript
    Study Types in Epidemiology Transcript Study Types in Epidemiology Welcome to “Study Types in Epidemiology.” My name is John Kobayashi. I’m on the Clinical Faculty at the Northwest Center for Public Health Practice, at the School of Public Health and Community Medicine, University of Washington in Seattle. From 1982 to 2001, I was the state epidemiologist for Communicable Diseases at the Washington State Department of Health. Since 2001, I’ve also been the foreign adviser for the Field Epidemiology Training Program of Japan. About this Module The modules in the epidemiology series from the Northwest Center for Public Health Practice are intended for people working in the field of public health who are not epidemiologists, but who would like to increase their understanding of the epidemiologic approach to health and disease. This module focuses on descriptive and analytic epide- miology and their respective study designs. Before you go on with this module, we recommend that you become familiar with the following modules, which you can find on the Center’s Web site: What is Epidemiology in Public Health? and Data Interpretation for Public Health Professionals. We introduce a number of new terms in this module. If you want to review their definitions, the glossary in the attachments link at the top of the screen may be useful. Objectives By now, you should be familiar with the overall approach of epidemiology, the use of various kinds of rates to measure disease frequency, and the various ways in which epidemiologic data can be presented. This module offers an overview of descriptive and analytic epidemiology and the types of studies used to review and investigate disease occurrence and causes.
    [Show full text]
  • Synthetic Control Method: Inference, Sensitivity Analysis and Confidence Sets∗
    Synthetic Control Method: Inference, Sensitivity Analysis and Confidence Sets∗ Sergio Firpo† Vitor Possebom‡ Insper Yale University September 1st, 2017 Abstract We extend the inference procedure for the synthetic control method in two ways. First, we impose a parametric form to the treatment assignment probabilities that includes, as a particular case, the uniform distribution that is assumed by Abadie et al.(2015). By changing the value of this parameter, we can analyze the sensitiveness of the test's decision to deviations from the usual uniform distribution assumption. Second, we modify the RMSPE statistic to test any sharp null hypothesis, including, as a specific case, the null hypothesis of no effect whatsoever analyzed by Abadie et al.(2015). Based on this last extension, we invert the test statistic to estimate confidence sets that quickly show the point-estimates' precision, and the test's significance and robustness. We also extend these two tools to other test statistics and to problems with multiple outcome variables or multiple treated units. Furthermore, in a Monte Carlo experiment, we find that the RMSPE statistic has good properties with respect to size, power and robustness. Finally, we illustrate the usefulness of our proposed tools by reanalyzing the economic impact of ETA's terrorism in the Basque Country, studied first by Abadie & Gardeazabal(2003) and Abadie et al.(2011). Keywords: Synthetic Control Estimator, Hypothesis Testing, Sensitivity Analysis, Con- fidence Sets JEL Codes: C21, C23, C33 ∗We are grateful to FAPESP that provided financial aid through grant number 2014/23731-3. We are in debt to the editor and two anonymous referees for useful suggestions.
    [Show full text]
  • Chapter 5 Experiments, Good And
    Chapter 5 Experiments, Good and Bad Point of both observational studies and designed experiments is to identify variable or set of variables, called explanatory variables, which are thought to predict outcome or response variable. Confounding between explanatory variables occurs when two or more explanatory variables are not separated and so it is not clear how much each explanatory variable contributes in prediction of response variable. Lurking variable is explanatory variable not considered in study but confounded with one or more explanatory variables in study. Confounding with lurking variables effectively reduced in randomized comparative experiments where subjects are assigned to treatments at random. Confounding with a (only one at a time) lurking variable reduced in observational studies by controlling for it by comparing matched groups. Consequently, experiments much more effec- tive than observed studies at detecting which explanatory variables cause differences in response. In both cases, statistically significant observed differences in average responses implies differences are \real", did not occur by chance alone. Exercise 5.1 (Experiments, Good and Bad) 1. Randomized comparative experiment: effect of temperature on mice rate of oxy- gen consumption. For example, mice rate of oxygen consumption 10.3 mL/sec when subjected to 10o F. temperature (Fo) 0 10 20 30 ROC (mL/sec) 9.7 10.3 11.2 14.0 (a) Explanatory variable considered in study is (choose one) i. temperature ii. rate of oxygen consumption iii. mice iv. mouse weight 25 26 Chapter 5. Experiments, Good and Bad (ATTENDANCE 3) (b) Response is (choose one) i. temperature ii. rate of oxygen consumption iii.
    [Show full text]
  • The Political Methodologist Newsletter of the Political Methodology Section American Political Science Association Volume 10, Number 2, Spring, 2002
    The Political Methodologist Newsletter of the Political Methodology Section American Political Science Association Volume 10, Number 2, Spring, 2002 Editor: Suzanna De Boef, Pennsylvania State University [email protected] Contents Summer 2002 at ICPSR . 27 Notes on the 35th Essex Summer Program . 29 Notes from the Editor 1 EITM Summer Training Institute Announcement 29 Note from the Editor of PA . 31 Teaching and Learning Graduate Methods 2 Michael Herron: Teaching Introductory Proba- Notes From the Editor bility Theory . 2 Eric Plutzer: First Things First . 4 This issue of TPM features articles on teaching the first Lawrence J. Grossback: Reflections from a Small course in the graduate methods sequence and on testing and Diverse Program . 6 theory with empirical methods. The first course presents Charles Tien: A Stealth Approach . 8 unique challenges: what to expect of students and where to begin. The contributions suggest that the first gradu- Christopher H. Achen: Advice for Students . 10 ate methods course varies greatly with respect to goals, content, and rigor across programs. Whatever the na- Testing Theory 12 ture of the course and whether you teach or are taking John Londregan: Political Theory and Political your first methods course, Chris Achen’s advice for stu- Reality . 12 dents beginning the methods sequence will be excellent reading. Rebecca Morton: EITM*: Experimental Impli- cations of Theoretical Models . 14 With the support of the National Science Foun- dation, the empirical implications of theoretical models (EITM) are the subject of increased attention. Given the Articles 16 empirical nature of the endeavor, EITM should inspire us. Andrew D. Martin: LATEX For the Rest of Us .
    [Show full text]
  • Randomized Controlled Trials, Development Economics and Policy Making in Developing Countries
    Randomized Controlled Trials, Development Economics and Policy Making in Developing Countries Esther Duflo Department of Economics, MIT Co-Director J-PAL [Joint work with Abhijit Banerjee and Michael Kremer] Randomized controlled trials have greatly expanded in the last two decades • Randomized controlled Trials were progressively accepted as a tool for policy evaluation in the US through many battles from the 1970s to the 1990s. • In development, the rapid growth starts after the mid 1990s – Kremer et al, studies on Kenya (1994) – PROGRESA experiment (1997) • Since 2000, the growth have been very rapid. J-PAL | THE ROLE OF RANDOMIZED EVALUATIONS IN INFORMING POLICY 2 Cameron et al (2016): RCT in development Figure 1: Number of Published RCTs 300 250 200 150 100 50 0 1975 1980 1985 1990 1995 2000 2005 2010 2015 Publication Year J-PAL | THE ROLE OF RANDOMIZED EVALUATIONS IN INFORMING POLICY 3 BREAD Affiliates doing RCT Figure 4. Fraction of BREAD Affiliates & Fellows with 1 or more RCTs 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 1980 or earlier 1981-1990 1991-2000 2001-2005 2006-today * Total Number of Fellows and Affiliates is 166. PhD Year J-PAL | THE ROLE OF RANDOMIZED EVALUATIONS IN INFORMING POLICY 4 Top Journals J-PAL | THE ROLE OF RANDOMIZED EVALUATIONS IN INFORMING POLICY 5 Many sectors, many countries J-PAL | THE ROLE OF RANDOMIZED EVALUATIONS IN INFORMING POLICY 6 Why have RCT had so much impact? • Focus on identification of causal effects (across the board) • Assessing External Validity • Observing Unobservables • Data collection • Iterative Experimentation • Unpack impacts J-PAL | THE ROLE OF RANDOMIZED EVALUATIONS IN INFORMING POLICY 7 Focus on Identification… across the board! • The key advantage of RCT was perceived to be a clear identification advantage • With RCT, since those who received a treatment are randomly selected in a relevant sample, any difference between treatment and control must be due to the treatment • Most criticisms of experiment also focus on limits to identification (imperfect randomization, attrition, etc.
    [Show full text]