DOCSLIB.ORG

Sweating the Small Stuff: Does Data Cleaning and Testing of Assumptions Really Matter in the 21St Century?

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Sweating the Small Stuff: Does Data Cleaning and Testing of Assumptions Really Matter in the 21St Century? SWEATING THE SMALL STUFF: DOES DATA CLEANING AND TESTING OF ASSUMPTIONS REALLY MATTER IN THE 21ST CENTURY? Topic Editor Jason W. Osborne PSYCHOLOGY FRONTIERS COPYRIGHT STATEMENT ABOUT FRONTIERS © Copyright 2007-2013 Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering Frontiers Media SA. approach to the world of academia, radically improving the way scholarly research is managed. All rights reserved. All content included on this site, The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share such as text, graphics, logos, button and generate knowledge. Frontiers provides immediate and permanent online open access to all icons, images, video/audio clips, downloads, data compilations and its publications, but this alone is not enough to realize our grand goals. software, is the property of or is licensed to Frontiers Media SA (“Frontiers”) or its licensees and/or subcontractors. The copyright in the FRONTIERS JOURNAL SERIES text of individual articles is the property of their respective authors, The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online subject to a license granted to Frontiers. journals, promising a paradigm shift from the current review, selection and dissemination The compilation of articles processes in academic publishing. constituting this e-book, as well as All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service all content on this site is the exclusive property of Frontiers. to the scholarly community. At the same time, the Frontiers Journal Series operates on a revo- Images and graphics not forming lutionary invention, the tiered publishing system, initially addressing specific communities of part of user-contributed materials may not be downloaded or copied scholars, and gradually climbing up to broader public understanding, thus serving the interests without permission. of the lay society, too. Articles and other user-contributed materials may be downloaded and reproduced subject to any copyright or other notices. No financial DEDICATION TO QUALITY payment or reward may be given for any such reproduction except to the Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interac- author(s) of the article concerned. tions between authors and review editors, who include some of the world’s best academicians. As author or other contributor you grant permission to others to Research must be certified by peers before entering a stream of knowledge that may eventually reproduce your articles, including reach the public - and shape society; therefore, Frontiers only applies the most rigorous and any graphics and third-party materials supplied by you, in unbiased reviews. accordance with the Conditions for Frontiers revolutionizes research publishing by freely delivering the most outstanding research, Website Use and subject to any copyright notices which you include evaluated with no bias from both the academic and social point of view. in connection with your articles and By applying the most advanced information technologies, Frontiers is catapulting scholarly materials. publishing into a new generation. All copyright, and all rights therein, are protected by national and international copyright laws. The above represents a summary WHAT ARE FRONTIERS RESEARCH TOPICS? only. For the full conditions see the Conditions for Authors and the Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are Conditions for Website Use. collections of at least ten articles, all centered on a particular subject. With their unique mix Cover image provided by Ibbl sarl, Lausanne CH of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot ISSN 1664-8714 research area! ISBN 978-2-88919-155-0 Find out more on how to host your own Frontiers Research Topic or contribute to one as an DOI 10.3389/978-2-88919-155-0 author by contacting the Frontiers Editorial Office: [email protected] Frontiers in Psychology August 2013 | Sweating the Small Stuff: Does data cleaning and testing of assumptions really matter in the 21st century? | 1 SWEATING THE SMALL STUFF: DOES DATA CLEANING AND TESTING OF ASSUMPTIONS REALLY MATTER IN THE 21ST CENTURY? Topic Editor: Jason W. Osborne, University of Louisville, USA Modern statistical software makes it easier than ever to do thorough data screening/cleaning and to test assumptions associated with the analyses researchers perform. However, few authors (even in top-tier journals) seem to be reporting data cleaning/screening and testing assumptions associated with the statistical analyses being reported. Few popular textbooks seem to focus on these basics. In the 21st Century, with our complex modern analyses, is data screening and cleaning still relevant? Do outliers or extreme scores matter any more? Does having normally distributed variables improve analyses? Are there new techniques for screening or cleaning data that researchers should be aware of? Are most analyses robust to violations of most assumptions, to the point that researchers really don’t need to pay attention to assumptions any more? My goal for this special issue is examine this issue with fresh eyes and 21st century methods. I believe that we can demonstrate that these things do still matter, even when using “robust” methods or non-parametric techniques, and perhaps identify when they matter MOST or in what way they can most substantially affect the results of an analysis. I believe we can encourage researchers to change their habits through evidence-based discussions revolving around these issues. It is possible we can even convince editors of important journals to include these aspects in their evaluation /review criteria, as many journals in the social sciences have done with effect size reporting in recent years. I invite you to join me in demonstrating WHY paying attention to these mundane aspects of quantitative analysis can be beneficial to researchers. Frontiers in Psychology August 2013 | Sweating the Small Stuff: Does data cleaning and testing of assumptions really matter in the 21st century? | 2 Table of Contents 05 Is Data Cleaning and the Testing of Assumptions Relevant in the 21st Century? Jason W. Osborne 08 Are Assumptions of Well-Known Statistical Techniques Checked, and Why (Not)? Rink Hoekstra, Henk A. L. Kiers and Addie Johnson 17 Statistical Conclusion Validity: Some Common Threats and Simple Remedies Miguel A. García-Pérez 28 Is Coefficient Alpha Robust to Non-Normal Data? Yanyan Sheng and Zhaohui Sheng 41 The Assumption of a Reliable Instrument and Other Pitfalls to Avoid When Considering the Reliability of Data Kim Nimon, Linda Reichwein Zientek and Robin K. Henson 54 Replication Unreliability in Psychology: Elusive Phenomena or “Elusive” Statistical Power? Patrizio E. Tressoldi 59 Distribution of Variables by Method of Outlier Detection W. Holmes Finch 71 Statistical Assumptions of Substantive Analyses Across the General Linear Model: A Mini-Review Kim F. Nimon 76 Tools to Support Interpreting Multiple Regression in the Face of Multicollinearity Amanda Kraha, Heather Turner, Kim Nimon, Linda Reichwein Zientek and Robin K. Henson 92 A Simple Statistic for Comparing Moderation of Slopes and Correlations Michael Smithson 101 Old and New Ideas for Data Screening and Assumption Testing for Exploratory and Confirmatory Factor Analysis David B. Flora, Cathy LaBrish and R. Philip Chalmers 122 On the Relevance of Assumptions Associated with Classical Factor Analytic Approaches† Daniel Kasper and Ali Ünlü Frontiers in Psychology August 2013 | Sweating the Small Stuff: Does data cleaning and testing of assumptions really matter in the 21st century? | 3 142 How Predictable are “Spontaneous Decisions” and “Hidden Intentions”? Comparing Classification Results Based on Previous Responses with Multivariate Pattern Analysis of fMRI BOLD Signals Martin Lages and Katarzyna Jaworska 150 Using Classroom Data to Teach Students About Data Cleaning and Testing Assumptions Kevin Cummiskey, Shonda Kuiper and Rodney Sturdivant Frontiers in Psychology August 2013 | Sweating the Small Stuff: Does data cleaning and testing of assumptions really matter in the 21st century? | 4 EDITORIAL published: 25 June 2013 doi: 10.3389/fpsyg.2013.00370 Is data cleaning and the testing of assumptions relevant in the 21st century? Jason W. Osborne* Educational and Counseling Psychology, University of Louisville, Louisville, Kentucky, USA *Correspondence: [email protected] Edited by: Axel Cleeremans, Université Libre de Bruxelles, Belgium You must understand fully what your assumptions say and what By the middle of the 20th century, researchers had assembled they imply. You must not claim that the “usual assumptions” are some evidence that some minimal violations of some assumptions acceptable due to the robustness of your technique unless you had minimal effects on error rates under certain circumstances— really understand the implications and limits of this assertion in in other words, if your variances are not exactly identical across the context of your application. And you must absolutely never all groups, but are relatively close, it is probably acceptable to use any statistical method without realizing that you are implic- interpret the results of that
Load more

Recommended publications
  • Case Study Applications of Statistics in Institutional Research
    /-- ; / \ \ CASE STUDY APPLICATIONS OF STATISTICS IN INSTITUTIONAL RESEARCH By MARY ANN COUGHLIN and MARIAN PAGAN( Case Study Applications of Statistics in Institutional Research by Mary Ann Coughlin and Marian Pagano Number Ten Resources in Institutional Research A JOINTPUBLICA TION OF THE ASSOCIATION FOR INSTITUTIONAL RESEARCH AND THE NORTHEASTASSO CIATION FOR INSTITUTIONAL REASEARCH © 1997 Association for Institutional Research 114 Stone Building Florida State University Ta llahassee, Florida 32306-3038 All Rights Reserved No portion of this book may be reproduced by any process, stored in a retrieval system, or transmitted in any form, or by any means, without the express written permission of the publisher. Printed in the United States To order additional copies, contact: AIR 114 Stone Building Florida State University Tallahassee FL 32306-3038 Tel: 904/644-4470 Fax: 904/644-8824 E-Mail: [email protected] Home Page: www.fsu.edul-airlhome.htm ISBN 1-882393-06-6 Table of Contents Acknowledgments •••••••••••••••••.•••••••••.....••••••••••••••••.••. Introduction .•••••••••••..•.•••••...•••••••.....••••.•••...••••••••• 1 Chapter 1: Basic Concepts ..•••••••...••••••...••••••••..••••••....... 3 Characteristics of Variables and Levels of Measurement ...................3 Descriptive Statistics ...............................................8 Probability, Sampling Theory and the Normal Distribution ................ 16 Chapter 2: Comparing Group Means: Are There Real Differences Between Av erage Faculty Salaries Across Departments? •...••••••••.••.••••••
    [Show full text]
  • Regression Assumptions in Clinical Psychology Research Practice—A Systematic Review of Common Misconceptions
    Regression assumptions in clinical psychology research practice—a systematic review of common misconceptions Anja F. Ernst and Casper J. Albers Heymans Institute for Psychological Research, University of Groningen, Groningen, The Netherlands ABSTRACT Misconceptions about the assumptions behind the standard linear regression model are widespread and dangerous. These lead to using linear regression when inappropriate, and to employing alternative procedures with less statistical power when unnecessary. Our systematic literature review investigated employment and reporting of assumption checks in twelve clinical psychology journals. Findings indicate that normality of the variables themselves, rather than of the errors, was wrongfully held for a necessary assumption in 4% of papers that use regression. Furthermore, 92% of all papers using linear regression were unclear about their assumption checks, violating APA- recommendations. This paper appeals for a heightened awareness for and increased transparency in the reporting of statistical assumption checking. Subjects Psychiatry and Psychology, Statistics Keywords Linear regression, Statistical assumptions, Literature review, Misconceptions about normality INTRODUCTION One of the most frequently employed models to express the influence of several predictors Submitted 18 November 2016 on a continuous outcome variable is the linear regression model: Accepted 17 April 2017 Published 16 May 2017 Yi D β0 Cβ1X1i Cβ2X2i C···CβpXpi C"i: Corresponding author Casper J. Albers, [email protected] This equation predicts the value of a case Yi with values Xji on the independent variables Academic editor Xj (j D 1;:::;p). The standard regression model takes Xj to be measured without error Shane Mueller (cf. Montgomery, Peck & Vining, 2012, p. 71). The various βj slopes are each a measure of Additional Information and association between the respective independent variable Xj and the dependent variable Y.
    [Show full text]
  • A Causation Coefficient and Taxonomy of Correlation/Causation Relationships
    A causation coefficient and taxonomy of correlation/causation relationships Joshua Brulé∗ Abstract This paper introduces a causation coefficient which is defined in terms of probabilistic causal models. This coefficient is suggested as the natural causal analogue of the Pearson correlation coefficient and permits comparing causation and correlation to each other in a simple, yet rigorous manner. Together, these coefficients provide a natural way to classify the possible correlation/causation relationships that can occur in practice and examples of each relationship are provided. In addition, the typical relationship between correlation and causation is analyzed to provide insight into why correlation and causation are often conflated. Finally, example calculations of the causation coefficient are shown on a real data set. Introduction The maxim, “Correlation is not causation”, is an important warning to analysts, but provides very little information about what causation is and how it relates to correlation. This has prompted other attempts at summarizing the relationship. For example, Tufte [1] suggests either, “Observed covariation is necessary but not sufficient for causality”, which is demonstrably false arXiv:1708.05069v1 [stat.ME] 5 Aug 2017 or, “Correlation is not causation but it is a hint”, which is correct, but still underspecified. In what sense is correlation a ‘hint’ to causation? Correlation is well understood and precisely defined. Generally speaking, correlation is any statistical relationship involving dependence, i.e. the ran- dom variables are not independent. More specifically, correlation can refer ∗Department of Computer Science, University of Maryland, College Park. [email protected] 1 to a descriptive statistic that summarizes the nature of the dependence.
    [Show full text]
  • The Quest for Statistical Significance: Ignorance, Bias and Malpractice of Research Practitioners Joshua Abah
    The Quest for Statistical Significance: Ignorance, Bias and Malpractice of Research Practitioners Joshua Abah To cite this version: Joshua Abah. The Quest for Statistical Significance: Ignorance, Bias and Malpractice of Research Practitioners. International Journal of Research & Review (www.ijrrjournal.com), 2018, 5 (3), pp.112- 129. hal-01758493 HAL Id: hal-01758493 https://hal.archives-ouvertes.fr/hal-01758493 Submitted on 4 Apr 2018 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Distributed under a Creative Commons Attribution - NonCommercial - ShareAlike| 4.0 International License International Journal of Research and Review www.gkpublication.in E-ISSN: 2349-9788; P-ISSN: 2454-2237 Review Article The Quest for Statistical Significance: Ignorance, Bias and Malpractice of Research Practitioners Joshua Abah Abah Department of Science Education University of Agriculture, Makurdi, Nigeria ABSTRACT There is a growing body of evidence on the prevalence of ignorance, biases and malpractice among researchers which questions the authenticity, validity and integrity of the knowledge been propagated in professional circles. The push for academic relevance and career advancement have driven some research practitioners into committing gross misconduct in the form of innocent ignorance, sloppiness, malicious intent and outright fraud.
    [Show full text]
  • Parametric and Nonparametric Bayesian Models for Ecological Inference in 2 × 2 Tables∗
    Parametric and Nonparametric Bayesian Models for Ecological Inference in 2 × 2 Tables∗ Kosuke Imai† Department of Politics, Princeton University Ying Lu‡ Office of Population Research, Princeton University November 18, 2004 Abstract The ecological inference problem arises when making inferences about individual behavior from aggregate data. Such a situation is frequently encountered in the social sciences and epi- demiology. In this article, we propose a Bayesian approach based on data augmentation. We formulate ecological inference in 2×2 tables as a missing data problem where only the weighted average of two unknown variables is observed. This framework directly incorporates the deter- ministic bounds, which contain all information available from the data, and allow researchers to incorporate the individual-level data whenever available. Within this general framework, we first develop a parametric model. We show that through the use of an EM algorithm, the model can formally quantify the effect of missing information on parameter estimation. This is an important diagnostic for evaluating the degree of aggregation effects. Next, we introduce a nonparametric model using a Dirichlet process prior to relax the distributional assumption of the parametric model. Through simulations and an empirical application, we evaluate the rela- tive performance of our models in various situations. We demonstrate that in typical ecological inference problems, the fraction of missing information often exceeds 50 percent. We also find that the nonparametric model generally outperforms the parametric model, although the latter gives reasonable in-sample predictions when the bounds are informative. C-code, along with an R interface, is publicly available for implementing our Markov chain Monte Carlo algorithms to fit the proposed models.
    [Show full text]
  • An Expert System for Designing Statistical
    AN EXPERT SYSTEM FOR DESIGNING STATISTICAL EXPERIMENTS. A thesis Presented to The Faculty of The College of Engineering And Technology Ohio University In Partial Fulfillment of The Requirements for the Degree of Nasters of Science Mustafa S. Shraim s- i ACKNOWLEDGEMENT I would like to express my thanks to my advisor, Dr. W. Robert Terry, for his help and encouragement throughout this research. Also, I would like to extend my thanks to my committee members, Dr. Francis Bellezza and Prof. Ken Cutright for their guidance and advice. This thesis is dedicated to my parents, Saadeddin and Etaf Shraim, for their continuous love and support and to my girlfriend, Ellen Battistone, without whom ..... TABLE OF CONTENTS 1.1 Experimental Design Review ------------------- 1 1.1.1 Basic Experimental Design Principles --- 2 1.1.2 Experimental Design Procedure ---------- 4 1.1.3 Statistical Techniques in Experimental Design .................... 5 1.2 Experimental Design and Quality Improvement -- 6 1.3 Statistical software 8 1.4 Knowledge Engineering Background ------------- 13 Chapter Two: NEEDS ANALYSIS .......................... 18 Chapter Three: TECHNICAL AND STATISTICAL TERMINOLOGY - 21 3.1 Definitions of Technical Terms 21 3.2 Definitions of Experimental Designs ---------- 26 3.3 Definitions of Tests ......................... 29 Chapter Four: EXPERIMENTAL DESIGN .................... 31 4.1 Introduction --------------------------------- 31 4.2 Basic Experimental Design Principles --------- 31 4.3 Screening Experiments ........................ 36 4.4 Model Adequacy ............................... 42 Chapter Five: KNOWLEDGE REPRESENTATION 50 5.1 Introduction ................................. 50 5.2 Experimental Design Rules .................... 51 5.2.1 Rules for Single Factor Experiments ---- 5 1 5.2.2 Rules for Factorial Experiments -------- 55 Chapter Six: SYSTEM EVALUATION ....................... 87 6.1 Introduction ................................
    [Show full text]
  • Guidance for Data Quality Assessment
    United States Office of Environmental EPA/600/R-96/084 Environmental Protection Information July, 2000 Agency Washington, DC 20460 Guidance for Data Quality Assessment Practical Methods for Data Analysis QualityEPA QA/G-9 QA00 UPDATE FOREWORD This document is the 2000 (QA00) version of the Guidance for Data Quality Assessment which provides general guidance to organizations on assessing data quality criteria and performance specifications for decision making. The Environmental Protection Agency (EPA) has developed a process for performing Data Quality Assessment (DQA) Process for project managers and planners to determine whether the type, quantity, and quality of data needed to support Agency decisions has been achieved. This guidance is the culmination of experiences in the design and statistical analyses of environmental data in different Program Offices at the EPA. Many elements of prior guidance, statistics, and scientific planning have been incorporated into this document. This document is distinctly different from other guidance documents; it is not intended to be read in a linear or continuous fashion. The intent of the document is for it to be used as a "tool-box" of useful techniques in assessing the quality of data. The overall structure of the document will enable the analyst to investigate many different problems using a systematic methodology. This document is one of a series of quality management guidance documents that the EPA Quality Staff has prepared to assist users in implementing the Agency-wide Quality System. Other related documents include: EPA QA/G-4 Guidance for the Data Quality Objectives Process EPA QA/G-4D DEFT Software for the Data Quality Objectives Process EPA QA/G-4HW Guidance for the Data Quality Objectives Process for Hazardous Waste Site Investigations EPA QA/G-9D Data Quality Evaluation Statistical Toolbox (DataQUEST) This document is intended to be a "living document" that will be updated periodically to incorporate new topics and revisions or refinements to existing procedures.
    [Show full text]
  • Statistical Assumptions of Substantive Analyses Across the General Linear Model: a Mini-Review
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE MINI REVIEW ARTICLE published: 28provided August by 2012 Frontiers - Publisher Connector doi: 10.3389/fpsyg.2012.00322 Statistical assumptions of substantive analyses across the general linear model: a mini-review Kim F. Nimon* Learning Technologies, University North Texas, Denton, TX, USA Edited by: The validity of inferences drawn from statistical test results depends on how well data Jason W. Osborne, Old Dominion meet associated assumptions.Yet, research (e.g., Hoekstra et al., 2012) indicates that such University, USA assumptions are rarely reported in literature and that some researchers might be unfamiliar Reviewed by: Anne C. Black, Yale University School with the techniques and remedies that are pertinent to the statistical tests they conduct. of Medicine, USA This article seeks to support researchers by concisely reviewing key statistical assumptions Megan Welsh, University of associated with substantive statistical tests across the general linear model. Additionally, Connecticut, USA the article reviews techniques to check for statistical assumptions and identifies remedies Cherng-Jyh Yen, Old Dominion University, USA and problems if data do not meet the necessary assumptions. *Correspondence: Keywords: assumptions, robustness, analyzing data, normality, homogeneity Kim F.Nimon, Learning Technologies, University North Texas, 3940 North Elm Street, G150 Denton, 76207 TX, USA. e-mail: [email protected] The degree to which valid inferences may be drawn from the researchers should describe the sample and population in suffi- results of inferential statistics depends upon the sampling tech- cient detail to justify that the sample was at least representative of nique and the characteristics of population data.
    [Show full text]
  • Conceptual Foundations: Pro B Ab Ility Distrib Utio Ns
    Analysis of Environmental Data Conceptual Foundations: Pro b ab ility Distrib utio ns 1. What is a probability (stochastic) distribution?. 2 1.1 Measurement error versus process error. 3 1.2 Conventional versus hierarchical models. 5 1.3 Probability distributions. 6 2. Discrete distributions. 7 2.1 Probability mass function.. 11 2.2 Cumulative probability distribution.. 12 2.3 Quantile distribution. 13 2.4 Random numbers. 14 2.5 Binomial examples. 15 3. Continuous probability distributions. 16 3.1 Probability density function. 19 3.2 Cumulative probability distribution.. 20 3.3 Quantile distribution. 21 3.4 Random numbers. 22 3.5 Normal (Gaussian) examples. 23 4. Bestiary of probability distributions. 24 4.1 Discrete distributions. 25 4.2 Continuous distributions. 29 5. Choosing the right probability distribution?. 37 6. Statistical Models – putting it all together. 38 Probability distributions 2 1. What is a probability (stochastic) distribution? Recall that the deterministic part of the statistical model describes the expected pattern in the absence of any kind of randomness or measurement error. The noise or error about the expected pattern is the stochastic component of the model. To formally estimate the parameters of the model, you need to know not just the expected pattern (the deterministic component) but also something about the variation about the expected pattern (the stochastic component). Typically, you describe the stochastic model by specifying a reasonable probability distribution for the variation about the expected pattern (see below). Probability distributions 3 1.1 So urc e s o f e rro r The variation about the expected pattern, i.e., the stochastic component of the model, is often termed “noise”.
    [Show full text]
  • Propensity Score Analysis Concepts and Issues
    ChapTER 1 Propensity Score Analysis Concepts and Issues Wei Pan Haiyan Bai ince the seminal paper by Rosenbaum and Rubin (1983b) on propen- Ssity score analysis, research using propensity score analysis has grown exponentially over three decades. Nevertheless, some methodological and practical issues still remain unresolved. This introductory chap- ter describes these issues along with an introduction to basic concepts of propensity score analysis. The remaining chapters in this book repre- sent a collective effort to further address these issues and provide demon- strations and recommendations on how to use propensity score analysis appropriately and effectively for various situations. The rest of this chapter proceeds as follows: It starts with a general introduction to propensity score analysis based on the theoretical frame- work of causal inference, followed by a detailed description of four major steps in propensity score analysis, and concludes with a brief discussion of current issues in propensity score analysis. CaUSAL INFERENCE AND PROPENSITY SCORE ANALYSIS Suppose one has N units (e.g., subjects). Denote z as a treatment condition and r as a potential response. For each unit i (i = 1, . , N), zi = 1 indicates that the unit i is in the treatment group with a corresponding potential 3 4 FUNDAMENTALS OF PROPENSITY SCORE ANALYSIS response r1i, and zi = 0 indicates that the unit i is in the control group with a corresponding potential response r0i. In the counterfactual framework for modeling causal effects (Holland, 1986; Rubin, 1974; Sobel, 1996; Win- ship & Morgan, 1999), the quantity of interest is the treatment effect for each unit i, which is defined as Di = r1i – r0i.
    [Show full text]
  • Part I: Statistical Design and Philosophy
    PART I Unified Guidance PART I. STATISTICAL DESIGN AND PHILOSOPHY Chapter 1 provides introductory information, including the purposes and goals of the guidance, as well as its potential applicability to other environmental programs. Chapter 2 presents a brief discussion of the existing regulations and identifies key portions of these rules which need to be addressed from a statistical standpoint, as well as some recommendations. In Chapter 3, fundamental statistical principles are highlighted which play a prominent role in the Unified Guidance including the notions of individual test false positive and negative decision errors and the accumulation of such errors across multiple tests or comparisons. Chapter 4 sets the groundwater monitoring program context, the nature of formal statistical tests for groundwater and some caveats in identifying statistically significant increases. Typical groundwater monitoring scenarios also are described in this chapter. Chapter 5 describes how to establish background and how to periodically update it. Chapters 6 and 7 outline various factors to be considered when designing a reasonable statistical strategy for use in detection monitoring, compliance/assessment monitoring, or corrective action. Finally, Chapter 8 summarizes the recommended statistical tests and methods, along with a concise review of assumptions, conditions of use, and limitations. March 2009 PART I Unified Guidance This page intentionally left blank March 2009 Chapter 1. Objectives Unified Guidance CHAPTER 1. OBJECTIVES AND POTENTIAL
    [Show full text]
  • Statistical Methods for Practitioners (EPA QA/G-9S)
    United States Office of Environmental Environmental Protection Information EPA/240/B-06/003 Agency Washington, DC 20460 February 2006 Data Quality Assessment: Statistical Methods for Practitioners EPA QA/G-9S FOREWORD This document, Data Quality Assessment: Statistical Methods for Practitioners, provides general guidance to organizations on assessing data quality criteria and performance specifications. The Environmental Protection Agency (EPA) has developed the Data Quality Assessment (DQA) Process for project managers and planners to determine whether the type, quantity, and quality of data needed to support Agency decision making has been achieved. This guidance is the culmination of experiences in the design and statistical analyses of environmental data in different Program Offices at the EPA. Many elements of prior guidance, statistics, and scientific planning have been incorporated into this document. This document is one of a series of quality management guidance documents that the EPA Quality Staff has prepared to assist users in implementing the Agency-wide Quality System. Other related documents include: EPA QA/G-4 Guidance on Systematic Planning Using the Data Quality Objectives Process EPA QA/G-5S Guidance on Choosing a Sampling Design for Environmental Data Collection EPA QA/G-9R Data Quality Assessment: A Reviewer’s Guide This document provides guidance to EPA program managers and planning teams as well as to the general public as appropriate. It does not impose legally binding requirements and may not apply to a particular situation based on the circumstances. EPA retains the discretion to adopt approaches on a case-by-case basis that differ from this guidance where appropriate. This guidance is one of the U.S Environmental Protection Agency Quality System Series documents.
    [Show full text]