DOCSLIB.ORG

Experimental Practices in Economics: a Methodological Challenge for Psychologists?

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Experimental Practices in Economics: a Methodological Challenge for Psychologists? BEHAVIORAL AND BRAIN SCIENCES (2001) 24, 383–451 Printed in the United States of America Experimental practices in economics: A methodological challenge for psychologists? Ralph Hertwig Center for Adaptive Behavior and Cognition, Max Planck Institute for Human Development, 14195 Berlin, Germany. [email protected] Andreas Ortmann Center for Economic Research and Graduate Education, Charles University, and Economics Institute, Academy of Sciences of the Czech Republic, 111 21 Prague 1, Czech Republic. [email protected] Abstract: This target article is concerned with the implications of the surprisingly different experimental practices in economics and in areas of psychology relevant to both economists and psychologists, such as behavioral decision making. We consider four features of ex- perimentation in economics, namely, script enactment, repeated trials, performance-based monetary payments, and the proscription against deception, and compare them to experimental practices in psychology, primarily in the area of behavioral decision making. Whereas economists bring a precisely defined “script” to experiments for participants to enact, psychologists often do not provide such a script, leaving participants to infer what choices the situation affords. By often using repeated experimental trials, economists allow participants to learn about the task and the environment; psychologists typically do not. Economists generally pay participants on the ba- sis of clearly defined performance criteria; psychologists usually pay a flat fee or grant a fixed amount of course credit. Economists vir- tually never deceive participants; psychologists, especially in some areas of inquiry, often do. We argue that experimental standards in economics are regulatory in that they allow for little variation between the experimental practices of individual researchers. The exper- imental standards in psychology, by contrast, are comparatively laissez-faire. We believe that the wider range of experimental practices in psychology reflects a lack of procedural regularity that may contribute to the variability of empirical findings in the research fields un- der consideration. We conclude with a call for more research on the consequences of methodological preferences, such as the use on monetary payments, and propose a “do-it-both-ways” rule regarding the enactment of scripts, repetition of trials, and performance-based monetary payments. We also argue, on pragmatic grounds, that the default practice should be not to deceive participants. Keywords: behavioral decision making; cognitive illusions; deception; experimental design; experimental economics; experimental prac- tices; financial incentives; learning; role playing 1. Introduction Ralph Hertwig is a research scientist at the Center for Adaptive Behavior and Cognition at the Max Planck Empirical tests of theories depend crucially on the method- Institute for Human Development in Berlin. Currently ological decisions researchers make in designing and im- he is a visiting scholar at Columbia University, New plementing the test (Duhem 1953; Quine 1953). Analyzing York. His research focuses on how people reason and and changing specific methodological practices, however, make decisions when faced with uncertainty, the role of can be a challenge. In psychology, for instance, “it is re- simple heuristics in human judgment and decision mak- markable that despite two decades of counterrevolutionary ing, and how heuristics are adapted to the ecological attacks, the mystifying doctrine of null hypothesis testing is structure of particular decision environments. In 1996, the German Psychological Association awarded him the still today the Bible from which our future research gener- Young Scientist Prize for his doctoral thesis. ation is taught” (Gigerenzer & Murray 1987, p. 27). Why is it so difficult to change scientists’ practices? One answer is Andreas Ortmann is an assistant professor at the Cen- that our methodological habits, rituals, and perhaps even ter for Economic Research and Graduate Education at quasi-religious attitudes about good experimentation are Charles University and researcher at the Economics Insti- tute of the Academy of Sciences of the Czech Republic, deeply entrenched in our daily routines as scientists, and both in Prague, and also a visiting research scientist at the hence often not reflected upon. Max Planck Institute for Human Development in Berlin. To put our practices into perspective and reflect on the An economist by training, his game-theoretic and experi- costs and benefits associated with them, it is useful to look mental work addresses the origins and evolution of lan- at methodological practices across time or across disci- guages, moral sentiments, conventions, and organizations. plines. Adopting mostly the latter perspective, in this arti- Downloaded© 2001 from https://www.cambridge.org/coreCambridge University Press. DSMZ 0140-525X/01, on 24 Mar 2020 $12.50 at 13:21:29, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms383. https://doi.org/10.1017/S0140525X01564146 Hertwig & Ortmann: Experimental practices in economics cle we point out that two related disciplines, experimental ticipant in a number of ways. In what follows, we consider economics and corresponding areas in psychology (in par- four key features of experimental practices in economics, ticular, behavioral decision making) have very different namely, script enactment, repeated trials, financial incen- conceptions of good experimentation. tives, and the proscription against deception. The differ- We discuss the different conceptions of good experi- ences between psychology and economics on these four mentation in terms of four key variables of experimental de- features can be summed up – albeit in a simplified way – sign and show how these variables tend to be realized dif- as follows. Whereas economists bring a precisely defined ferently in the two disciplines. In addition, we show that “script” to experiments and have participants enact it, psy- experimental standards in economics, such as performance- chologists often do not provide such a script. Economists based monetary payments (henceforth, financial incen- often repeat experimental trials; psychologists typically do tives) and the proscription against deception, are rigorously not. Economists almost always pay participants according enforced through conventions or third parties. As a result, to clearly defined performance criteria; psychologists usu- these standards allow for little variation in the experimen- ally pay a flat fee or grant a fixed amount of course credit. tal practices of individual researchers. The experimental Economists do not deceive participants; psychologists, par- standards in psychology, by contrast, are comparatively lais- ticularly in social psychology, often do. sez-faire, allowing for a wider range of practices. The lack We argue that economists’ realizations of these variables of procedural regularity and the imprecisely specified social of experimental design reduce participants’ uncertainty by situation “experiment” that results may help to explain why explicitly stating action choices (script), allowing partici- in the “muddy vineyards” (Rosenthal 1990, p. 775) of soft pants to gain experience with the situation (repeated trials), psychology, empirical results “seem ephemeral and un- making clear that the goal is to perform as well as they can replicable” (p. 775). (financial incentives), and limiting second-guessing about the purpose of the experiment (no deception). In contrast, psychologists’ realizations of these variables tend to allow 1.1. The uncertain meaning of the social more room for uncertainty by leaving it unclear what the situation “experiment” action choices are (no script), affording little opportunity In his book on the historical origins of psychological exper- for learning (no repeated trials), leaving it unclear what the imentation, Danziger (1990) concluded that “until rela- experimenters want (no financial incentives), and prompt- tively recently the total blindness of psychological investi- ing participants to second-guess (deception). gators to the social features of their investigative situations Before we explore these differences in detail, four ca- constituted one of the most characteristic features of their veats are in order. First, the four variables of experimental research practice” (p. 8). This is deplorable because the ex- design we discuss are, in our view, particularly important perimenter and the human data source are necessarily en- design variables. This does not mean that we consider oth- gaged in a social relationship; therefore, experimental re- ers to be irrelevant. For example, we question economists’ sults in psychology will always be codetermined by the usual assumption that the abstract laboratory environment social relationship between experimenter and participant. in their experiments is neutral and, drawing on results from Schultz (1969) observed that this relationship “has some of cognitive psychology, have argued this point elsewhere the characteristics of a superior-subordinate one.... Per- (Ortmann & Gigerenzer 1997). Second, we stress that haps the only other such one-sided relationships are those whenever we speak of standard experimental practices in of parent and child, physician and patient, or drill sergeant “psychology,” we mean those used in research on behavioral and trainee” (p. 221). The
Load more

Recommended publications
  • Working Memory, Cognitive Miserliness and Logic As Predictors of Performance on the Cognitive Reflection Test
    Working Memory, Cognitive Miserliness and Logic as Predictors of Performance on the Cognitive Reflection Test Edward J. N. Stupple ([email protected]) Centre for Psychological Research, University of Derby Kedleston Road, Derby. DE22 1GB Maggie Gale ([email protected]) Centre for Psychological Research, University of Derby Kedleston Road, Derby. DE22 1GB Christopher R. Richmond ([email protected]) Centre for Psychological Research, University of Derby Kedleston Road, Derby. DE22 1GB Abstract Most participants respond that the answer is 10 cents; however, a slower and more analytic approach to the The Cognitive Reflection Test (CRT) was devised to measure problem reveals the correct answer to be 5 cents. the inhibition of heuristic responses to favour analytic ones. The CRT has been a spectacular success, attracting more Toplak, West and Stanovich (2011) demonstrated that the than 100 citations in 2012 alone (Scopus). This may be in CRT was a powerful predictor of heuristics and biases task part due to the ease of administration; with only three items performance - proposing it as a metric of the cognitive miserliness central to dual process theories of thinking. This and no requirement for expensive equipment, the practical thesis was examined using reasoning response-times, advantages are considerable. There have, moreover, been normative responses from two reasoning tasks and working numerous correlates of the CRT demonstrated, from a wide memory capacity (WMC) to predict individual differences in range of tasks in the heuristics and biases literature (Toplak performance on the CRT. These data offered limited support et al., 2011) to risk aversion and SAT scores (Frederick, for the view of miserliness as the primary factor in the CRT.
    [Show full text]
  • Independent Research Group 234
    Impressum © 2003 Max Planck Institute for Human Development, Berlin Design: Grafisches Atelier Rudolf J. Schmitt, Berlin Realization: Jürgen Baumgarten, Ivonne Bratke, Renate Hoffmann, Ulrich Kuhnert, Yvonne Misun, Erna Schiwietz, Peter Wittek of the Max Planck Institute for Human Development Printed 2003 by DruckVerlag Kettler GmbH, Bönen/Westfalen, Germany Board of Directors Paul B. Baltes Jürgen Baumert Gerd Gigerenzer (Managing Director, January–December 2001) Karl Ulrich Mayer (Managing Director, January–December 2002) Board of External Scientific Advisers Marlis Buchmann Laura L. Carstensen Leda Cosmides Jacquelynne S. Eccles Klaus Fiedler Andreas Krapp Herbert W. Marsh Walter Müller Jürgen Oelkers Anik de Ribaupierre-Bobillier Contents Introduction 6 Highlights 12 Cooperation with Universities 16 Center for Adaptive Behavior and Cognition 24 Center for Educational Research 60 Center for Lifespan Psychology 128 Center for Sociology and the Study of the Life Course 188 Independent Research Group 234 Service Units 248 Appendix 256 Introduction 8 Introduction Introduction The Max Planck Institute for Human Development is a multidisciplinary re- search establishment dedicated to the study of human development and ed- ucation. Its inquiries are broadly defined, but concentrate on the evolution- ary, social, historical, and institutional contexts of human development, as well as examining it from life-span and life-course perspectives. The disci- plines of education, psychology, and sociology reflect the current directors’ backgrounds, but the Institute’s scholarly spectrum is enriched by the work of colleagues from such fields as mathematics, economics, computer sci- ence, evolutionary biology, and the humanities. The Institute is one of about 80 research facilities financed by the Max Planck Society for the Advancement of Science (Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V.), the core support for which is pro- vided by the Federal Republic of Germany and its 16 states.
    [Show full text]
  • Maria-Pia Victoria-Feser GSEM, University of Geneva B [email protected] Professor of Statistics
    Research Center for Statistics Maria-Pia Victoria-Feser GSEM, University of Geneva B [email protected] Professor of Statistics Executive Summary (September 2019) Employment I started my career as a lecturer at the London School of Economics (LSE) in 1993, the year I completed history my Ph. D. in econometrics and statistics. In 1997, I was hired as lecturer in the Faculty of Psychology and Educational Sciences (FPSE) of the University of Geneva. Between 1997 and 2001, my position has evolved until I became full professor (50%) in 2001 at HEC-Genève. Since 2018, I have a 100% permanent position at the Geneva School of Economics and Management (GSEM). Awards and I received a total of 3,277,537 CHF of personal research funding (Swiss National Science Foundation grants (SNSF) and private) for seven projects, and over 500,000 CHF of research funding for collaborative projects. I was awarded the Latzis International Prize (1995) for my Ph. D. Thesis, as well as a doctoral (1991) and a professorial (2000) fellowships from the SNSF. Research My main research interest is statistical methodology and its application in different domains. I started with the development of robust methods of estimation and inference for application in economics (welfare analysis, risk analysis) and psychology (psychometric). I am also interested in computational statistics and have developed estimation methods for complex models (latent dependence structures, missing data), time series (signal processing), and model selection in high dimensions. My publications are often co-authored with former Ph. D. students or with academic researchers in other universities and/or outside the methodological statistics field.
    [Show full text]
  • Opting-In: Participation Bias in Economic Experiments*
    Opting-in: Participation Bias in Economic Experiments* Robert Slonim** School of Economics, Faculty of Arts and Social Sciences, University of Sydney and IZA Carmen Wang Harvard Business School Ellen Garbarino Discipline of Marketing, Business School, University of Sydney Danielle Merrett The World Bank, Sydney, Australia. Abstract Assuming individuals rationally decide whether to participate or not to participate in lab experiments, we hypothesize several non-representative biases in the characteristics of lab participants. We test the hypotheses by first collecting survey and experimental data from a typical recruitment population and then inviting them to participate in a lab experiment. The results indicate that lab participants are not representative of the target population on almost all the hypothesized characteristics, including having lower income, working fewer hours, volunteering more often, and exhibiting behaviors correlated with interest in experiments and economics. The results reinforce the commonly understood limits of laboratory research to make quantitative inferences. We also discuss several methods for addressing non-representative biases to advance laboratory methods for improving quantitative inferences and consequently increasing confidence in qualitative conclusions. Key words: Participation Bias, Laboratory Experiments *Acknowledgements: We have greatly benefitted from numerous seminar participant comments at the University of Sydney, the Winter School on Experimental Economics at the University of Sydney, the University of Technology, Sydney, the University of New South Wales and the 6th annual Australia-New Zealand Workshop in Experimental Economics. We also appreciate comments and suggestions from Paul Frijters, Simon Gaechter, Ben Greiner, Nickolas Feltovich, Nikos Nikiforakis, Andreas Ortmann, the editor and two anonymous reviewers at the Journal of Economic Behavior and Organization.
    [Show full text]
  • UCLA Electronic Theses and Dissertations
    UCLA UCLA Electronic Theses and Dissertations Title Bias Analyses in Population-Based Studies of Parkinson’s Disease Permalink https://escholarship.org/uc/item/37k4j1j0 Author Cui, Xin Publication Date 2015 Peer reviewed|Thesis/dissertation eScholarship.org Powered by the California Digital Library University of California UNIVERSITY OF CALIFORNIA Los Angeles Bias Analyses in Population-Based Studies of Parkinson’s Disease A dissertation submitted in partial satisfaction of the requirements for the degree Doctor of Philosophy in Epidemiology by Xin Cui 2015 - 1 - © Copyright by Xin Cui 2015 - 2 - ABSTRACT OF THE DISSERTATION Bias Analyses in Population-Based Studies of Parkinson’s Disease by Xin Cui Doctor of Philosophy in Epidemiology University of California, Los Angeles, 2015 Professor Beate Ritz, Chair This dissertation work investigates two types of biases in population-based studies of Parkinson’s disease (PD), one is survivor bias in estimating the association between PD and cancer, the other is exposure misclassification due to residential mobility and time-varying exposure. Negative associations between PD and cancer have been found in epidemiological studies. Several mechanisms were proposed to explain such reported inverse associations. In the first and second part of this work we propose two similar survivor bias mechanisms that may account for the observed negative associations with cancer both prior to and after the diagnosis of PD as an alternative explanation. Using a large Danish population-based case-control study of ii Parkinson’s disease as an example, we utilize causal theory, Monte Carlo methods and inverse probability-of-censoring weights techniques to quantitatively investigate how the observed negative association can be explained by the hypothesized bias.
    [Show full text]
  • Biases in Machine Learning Models and Big Data Analytics: the International Criminal and Humanitarian Law Implications Nema Milaninia*
    International Review of the Red Cross (2020), 102 (913), 199–234. Digital technologies and war doi:10.1017/S1816383121000096 Biases in machine learning models and big data analytics: The international criminal and humanitarian law implications Nema Milaninia* Abstract Advances in mobile phone technology and social media have created a world where the volume of information generated and shared is outpacing the ability of humans to review and use that data. Machine learning (ML) models and “big data” analytical tools have the power to ease that burden by making sense of this information and providing insights that might not otherwise exist. In the context of international criminal and human rights law, ML is being used for a variety of purposes, including to uncover mass graves in Mexico, find evidence of homes and schools destroyed in Darfur, detect fake videos and doctored evidence, predict the outcomes of judicial hearings at the European Court of Human Rights, and gather evidence of war crimes in Syria. ML models are also increasingly being incorporated by States into weapon systems in order to better enable targeting * The views expressed herein are those of the author alone and do not necessarily reflect the views of the ICC Office of the Prosecutor or Google. The author would like to thank Nasrina Bargzie, Alexa Koenig, Matthew Cross, Beth Van Schaack, María Sol Beker, Jérôme de Hemptinne, Nikki Ahmadi, Gayane Khechoomian, Yulia Nuzban, and the editors of the International Review of the Red Cross, Bruno Demeyere, Sai Venkatesh and Ash Stanley-Ryan, for their considerably important and insightful input and feedback.
    [Show full text]
  • Examining the Effects of Gender and Written Communication Style on Task Completion
    St. Cloud State University theRepository at St. Cloud State Culminating Projects in Economics Department of Economics 3-2021 To Perform or Not to Perform? Examining the Effects of Gender and Written Communication Style on Task Completion Alvaro Plachejo [email protected] Follow this and additional works at: https://repository.stcloudstate.edu/econ_etds Recommended Citation Plachejo, Alvaro, "To Perform or Not to Perform? Examining the Effects of Gender and Written Communication Style on Task Completion" (2021). Culminating Projects in Economics. 17. https://repository.stcloudstate.edu/econ_etds/17 This Thesis is brought to you for free and open access by the Department of Economics at theRepository at St. Cloud State. It has been accepted for inclusion in Culminating Projects in Economics by an authorized administrator of theRepository at St. Cloud State. For more information, please contact [email protected]. To Perform or Not to Perform? Examining the Effects of Gender and Written Communication Style on Task Completion By Álvaro Plachejo A Thesis Submitted to the Graduate Faculty of St. Cloud State University in Partial Fulfillment of the Requirements for the Degree of Master of Science in Applied Economics March, 2021 Thesis Committee: Mana Komai Molle, Chairperson Patricia Hughes Susan Parault Dowds 2 Abstract I study the effect of two power categories, gender (male/female) and written communication style (strong language/weak language) on performance. To examine this relationship, these two attributes are considered in a request to perform a task using an experimental design on Amazon Mechanical Turk. Three variations of the experiment are performed: announcing an additional monetary reward for task performance, not announcing the reward, and explicitly warning that no reward is provided.
    [Show full text]
  • Research Participation Effects: a Skeleton in the Methodological
    Journal of Clinical Epidemiology - (2014) - COMMENTARY Research participation effects: a skeleton in the methodological cupboard* Jim McCambridgea,*, Kypros Kyprib, Diana Elbournec aFaculty of Public Health & Policy, London School of Hygiene & Tropical Medicine, 15-17 Tavistock Place, London WC1H 9SH, UK bCentre for ClinicalEpidemiologyand Biostatistics, School ofMedicineand Public Health, University of Newcastle, HMRIBuilding,Callaghan NSW2308, Australia cDepartment of Medical Statistics, Faculty of Epidemiology and Population Health, London School of Hygiene & Tropical Medicine, Keppel Street, London WC1E 7HT, UK Accepted 14 March 2014; Published online xxxx Abstract Objective: There have been concerns about impacts of various aspects of taking part in research studies for a century. The concerns have not, however, been sufficiently well conceptualized to form traditions of study capable of defining and elaborating the nature of these problems. In this article we present a new way of thinking about a set of issues attracting long-standing attention. Study Design and Setting: We briefly review existing concepts and empirical work on well-known biases in surveys and cohort studies and propose that they are connected. Results: We offer the construct of ‘‘research participation effects’’ (RPE) as a vehicle for advancing multi-disciplinary understanding of biases. Empirical studies are needed to identify conditions in which RPE may be sufficiently large to warrant modifications of study design, analytic methods, or interpretation. We consider the value of adopting a more participant-centred view of the research process as a way of thinking about these issues, which may also have benefits in relation to research methodology more broadly. Conclusion: Researchers may too readily overlook the extent to which research studies are unusual contexts, and that people may react in unexpected ways towhat we invite them to do, introducing a range of biases.
    [Show full text]
  • Creating Disorientation, Fugue, And
    OCCLUSION: CREATING DISORIENTATION, FUGUE, AND APOPHENIA IN AN ART GAME by Klew Williams A Thesis Submitted to the Faculty of the WORCESTER POLYTECHNIC INSTITUTE in partial fulfillment of the requirements for the Degree of Master of Science in Interactive Media and Game Development __________________________________________________________ April 27th, 2017 APPROVED: ____________________________________ Brian Moriarty, Thesis Advisor _____________________________________ Dean O’Donnell, Committee _____________________________________ Ralph Sutter, Committee i Abstract Occlusion is a procedurally randomized interactive art experience which uses the motifs of repetition, isolation, incongruity and mutability to develop an experience of a Folie à Deux: a madness shared by two. It draws from traditional video game forms, development methods, and tools to situate itself in context with games as well as other forms of interactive digital media. In this way, Occlusion approaches the making of game-like media from the art criticism perspective of Materiality, and the written work accompanying the prototype discusses critical aesthetic concerns for Occlusion both as an art experience borrowing from games and as a text that can be academically understood in relation to other practices of media making. In addition to the produced software artifact and written analysis, this thesis includes primary research in the form of four interviews with artists, authors, game makers and game critics concerning Materiality and dissociative themes in game-like media. The written work first introduces Occlusion in context with other approaches to procedural remixing, Glitch Art, net.art, and analogue and digital collage and décollage, with special attention to recontextualization and apophenia. The experience, visual, and audio design approach of Occlusion is reviewed through a discussion of explicit design choices which define generative space.
    [Show full text]
  • Protecting Against Researcher Bias in Secondary Data Analysis
    Protecting against researcher bias in secondary data analysis: Challenges and potential solutions Jessie R. Baldwin1,2, PhD, Jean-Baptiste Pingault1,2, PhD, Tabea Schoeler,1 PhD, Hannah M. Sallis3,4,5, PhD & Marcus R. Munafò3,4,6, PhD 4,618 words; 2 tables; 1 figure 1 Department of Clinical, Educational and Health Psychology, Division of Psychology and Language Sciences, University College London, London, UK 2 Social, Genetic and Developmental Psychiatry Centre, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, UK 3 MRC Integrative Epidemiology Unit at the University of Bristol, Bristol Medical School, University of Bristol, Bristol, UK 4 School of Psychological Science, University of Bristol, Bristol, UK 5 Centre for Academic Mental Health, Population Health Sciences, University of Bristol, Bristol, UK 6 NIHR Biomedical Research Centre, University Hospitals Bristol NHS Foundation Trust and University of Bristol, Bristol, UK Correspondence: Dr. Jessie R. Baldwin, Department of Clinical, Educational and Health Psychology, Division of Psychology and Language Sciences, University College London, London, WC1H 0AP, UK; [email protected] Funding: J.R.B is funded by a Wellcome Trust Sir Henry Wellcome fellowship (grant 215917/Z/19/Z). J.B.P is a supported by the Medical Research Foundation 2018 Emerging Leaders 1st Prize in Adolescent Mental Health (MRF-160-0002-ELP-PINGA). M.R.M and H.M.S work in a unit that receives funding from the University of Bristol and the UK Medical Research Council (MC_UU_00011/5, MC_UU_00011/7), and M.R.M is also supported by the National Institute for Health Research (NIHR) Biomedical Research Centre at the University Hospitals Bristol National Health Service Foundation Trust and the University of Bristol.
    [Show full text]
  • Field Experiments in Economics: Some Methodological Caveats
    Field Experiments in Economics: Some Methodological Caveats by ANDREAS ORTMANN Center for Economic Research and Graduate Education Charles University (CERGE) and Economics Institute (EI) Academy of Sciences of the Czech Republic Prague, Czech Republic July 31, 2003 CERGE-EI P.O.BOX 882, Politickych veznu 7, 111 21 Prague, Czech Republic tel.: (420-2) 242 30 117, fax: (420-2) 242 11 374, 242 27 143 e-mail:[email protected] [email protected] [email protected] DRAFT; DON’T QUOTE: DRAFT; DON’T QUOTE: DRAFT; DON’T QUOTE: * Thanks to the organizers and participants of the Middlebury Conference on Field Experiments in Economics (April 26-27, 2003). Special thanks to Glenn Harrison, Ralph Hertwig, and Angelika Weber for comments on earlier versions of this manuscript. The usual caveats apply. Abstract The results of standard lab experiments have long been questioned because of the convenience sample of subjects – college students – they typically employ and the abstract nature of the typical lab setting. These procedural conventions of experimental economics, it is argued, endanger the external validity of experiments. Various forms of field experiments (see Harrison and List 2003a for a taxonomy) have tried to address these issues by bringing the lab to non-traditional subjects (including participants in exotic locales), and/or to move the setting of experiments closer to reality by using real goods and settings that are not stripped of context. While field experiments might help experimental economists to broaden their subject pools, and to increase the external validity of their investigations, I argue that these potential advantages do come at costs that can be considerable.
    [Show full text]
  • Academia and Clinic
    The STROBE Statement: Explanation and Elaboration Academia and Clinic universally agreed definitions for participation, response, or likely to be related to health status (for example, the letter follow-up rates, readers need to understand how authors of invitation was not delivered because of an incorrect ad- calculated such proportions (134). dress) will affect the precision of estimates but will proba- Ideally, investigators should give an account of the bly not introduce bias. Conversely, if many individuals opt numbers of individuals considered at each stage of recruit- out of the survey because of illness or perceived good ing study participants, from the choice of a target popula- health, results may underestimate or overestimate the prev- tion to the inclusion of participants’ data in the analysis. alence of ill health in the population. Depending on the type of study, this may include the number of individuals considered to be potentially eligible, 13(c) Consider use of a flow diagram. the number assessed for eligibility, the number found to be eligible, the number included in the study, the number Example examined, the number followed up, and the number in- See the Figure. cluded in the analysis. Information on different sampling units may be required, if sampling of study participants is carried out in 2 or more stages as in the example above Explanation (multistage sampling). In case–control studies, we advise An informative and well-structured flow diagram can that authors describe the flow of participants separately for readily and transparently convey information that might case and control groups (135). Controls can sometimes be otherwise require a lengthy description (142), as in the selected from several sources, including, for example, hos- example above.
    [Show full text]