DOCSLIB.ORG

Fast Implementation of Delong's Algorithm for Comparing the Areas Under Correlated Receiver Operating Characteristic Curves

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Fast Implementation of Delong's Algorithm for Comparing the Areas Under Correlated Receiver Operating Characteristic Curves MANUSCRIPT ACCEPTED BY IEEE SIGNAL PROCESSING LETTERS 1 Fast Implementation of DeLong’s Algorithm for Comparing the Areas Under Correlated Receiver Operating Characteristic Curves Xu Sun; Weichao Xu∗, Member, IEEE Abstract—Among algorithms for comparing the areas under directly from the samples without making any parametric two or more correlated receiver operating characteristic (ROC) model assumptions on the forms of the parent populations. curves, DeLong’s algorithm is perhaps the most widely used For samples drawn from continuous distributions, algorithms one due to its simplicity of implementation in practice. Unfor- tunately, however, the time complexity of DeLong’s algorithm is of linearithmic time complexity have been proposed by Xu et of quadratic order (the product of sample sizes), thus making it al. for estimating the mean and variance of AUC [17]. Unfortu- time-consuming and impractical when the sample sizes are large. nately, however, sometimes in practice, the samples obey non- Based on an equivalent relationship between the Heaviside func- continuous distributions, that is, the probability of ties between tion and mid-ranks of samples, we improve DeLong’s algorithm samples are not zero. Under this circumstance, sub-quadratic by reducing the order of time complexity from quadratic down to linearithmic (the product of sample size and its logarithm). estimators for the variance of and covariance between AUCs Monte Carlo simulations verify the computational efficiency of are still unavailable to the best of our knowledge. This lack of our algorithmic findings in this work. efficient algorithms makes AUC comparison computationally Index Terms—Area under the curve (AUC), DeLong’s method, very expensive in scenarios (e.g. bioinformatics) involving mid-rank, receiver operating characteristic (ROC). massive data analysis. Motivated by this unsatisfactory situation, in this work we I. INTRODUCTION improve the popular DeLong’s algorithm [14] by reducing the time complexity from quadratic down to linearithmic order. RIGINATED from detection theory developed during This is accomplished through a relationship we find between World War II [1]–[4], receiver operating characteris- the Heaviside function and the mid-ranks of samples. ticO (ROC) analysis has found a wide use in a number of The rest part of this paper is organized as follows. Section II fields, including medicine, psychology, bioinformatics, signal gives the basic definition of AUC as well as some general processing, and machine learning, just to name a few [5]– notations employed throughout this work. Section III depicts [9]. Geometrically, ROC curve is a two-dimensional curve our linearithmic algorithm after establishing a close relation- traced out by pairs of false-positive rate and true-positive ship between the Heaviside function and mid-ranks associated rate according to various decision threshold settings. Given with samples. In Section IV, we demonstrate the efficiency of the ROC curve, the area under the curve (AUC) can then our improved algorithm in terms of time-complexity by Monte be computed, either analytically or empirically, as a figure Carlo experiments. Finally, we summarize our main finding of merit to summarize a diagnostic system’s performance [5], and draw our conclusion in Section V. a binary classifier’s overall accuracy [6], or a detector’s power of detecting the presence of an unknown signal [7]–[9]. II. DEFINITIONS As revealed by Bamber [10], AUC can be estimated by the Mann-Whitney U statistic (MWUS) [11]. Based on such For completeness and ease of later development, this section relationship, a plenty of nonparametric methods have been pro- describes the definition of nonparametric estimator of AUC as posed in the literature [12]–[16], which formulate algorithms well as DeLong’s formulas for estimating the variance of and covariance between correlated AUCs. Copyright (c) 2012 IEEE. Personal use of this material is permitted. Let X1;:::;Xm and Y1;:::;Yn be two independent and However, permission to use this material for any other purposes must be obtained from the IEEE by sending a request to [email protected]. identically distributed (i.i.d) samples drawn from two popula- This work was supported in part by National Natural Science Foundation of tions (whose distributions can be either continuous or discrete). China (Project 61271380), in part by Guangdong Natural Science Foundation Then, based on the relationship between the MWUS and AUC, (Project S2012010009870), in part by 100-Talents Scheme Funding from Guangdong University of Technology (Grant 112418006), in part by the Talent the sample version of AUC can be defined as Introduction Special Funds from Guangdong Province (Grant 2050205), in 1 m n part by a team project from Guangdong University of Technology (Grant ^ θ , (Xi Yj) (1) GDUT2011-07), and in part by Project Program of Key Laboratory of mn H − i=1 j=1 Guangdong Higher Education Institutes of China (Grant 2013CXZDA015). X X X, Sun and W. Xu are with the Department of Automatic Control, School where of Automation, Guangdong University of Technology, Guangzhou, 510006, P. R. China (e-mail:[email protected]; [email protected]). 1 t > 0 ∗Corresponding Author. (t) = 1 t = 0 (2) Tel:+86-020-39322552 Fax:+86-020-39322469 H 8 2 EDICS:MLSAS-PATT; SAS-MALN; SAS-STAT < 0 t < 0 : 2 MANUSCRIPT ACCEPTED BY IEEE SIGNAL PROCESSING LETTERS is the familiar Heaviside function. A. Relationship between mid-ranks and ( ) H · Let E( ), V( ) and C( ; ) denote the mean, variance and Let ;:::; be a sequence of real numbers. Sorting the · · · · Z1 ZM covariance of (between) random variables, respectively. It is sequence in ascending order yields a new sequence, termed ^ easily seen from (1) that θ is an unbiased estimator of the the order statistics [18]–[23], as corresponding population version θ, since = = < < = = (= ) Z(1) ··· Z(1) ··· Z(J) ··· Z(J) Zi 1 ^ Block1 BlockJ E(θ) = θ , Pr(X > Y ) + Pr(X = Y ): (3) (10) 2 < < = = : | {z } ··· Z| (K) ··· {z Z(K) } Let BlockK ^ ^(1) ^(k) θ , θ ;:::; θ (4) Suppose that BlockJ , whose elements are all equal to i, starts f g | {z } Z at position a and ends at position b in the sorted sequence (10). be a vector of statistics representing the areas under the ROC Then the mid-ranks of i’s in the original sequence is defined (r) (r) Z curves derived from different readings X1 ;:::;Xm and as [24] (r) (r) b Y1 ;:::;Yn (1 r k) of k different experiments. 1 a + b ≤ ≤ T ( i) = k = : (11) For the rth element of the vector, define the “structural Z Z b a + 1 2 k=a components” − X Given (11), it then follows that mid-ranks are closed related to 1 n the Heaviside function ( ), as shown in the following lemma. V (X(r)) = (X(r) Y (r)); i = 1; : : : ; m (5) H · 10 i n H i − j Lemma 1: The mid-ranks of i’s in 1;:::; M can be j=1 Z Z Z X computed by and M 1 T ( i) = ( i j) + : (12) m Z Z H Z − Z 2 1 j=1 V (Y (r)) = (X(r) Y (r)); j = 1; : : : ; n: (6) X 01 j m H i − j i=1 Proof: It suffices to show that the right side of (12) equals X the rightmost term in (11), as (r;s) Also define two matrices S 10 , s10 and S 01 , M a 1 b k k 1 − 1 1 a + b × ( i j) + = 1 + 1 + = : (13) s(r;s) such that h i H Z − Z 2 2 2 2 01 j=1 k k X kX=1 kX=a h i × 1 m (r;s) (r) ^(r) (s) ^(s) Note that from (10) and (11), T ( i), i = 1;:::;M, can be s10 = V10(Xi ) θ V10(Xi ) θ Z Z m 1 − − obtained in linearithmic time, i.e., (M log M), by using the − i=1 O X h i h (7)i popular quick sort algorithm [25] (See Fig. 1). The relationship and of (12), while useful for developing efficient algorithms later on, is not employed to calculate T ( i), i = 1;:::;M, due 1 n Z Z s(r;s) = V (Y (r)) θ^(r) V (Y (s)) θ^(s) : to its quadratic time complexity. 01 n 1 01 j − 01 j − j=1 − X h i h i (8) B. Fast algorithm for computing θ^ Then, DeLong et al. [14] proposed a variance-covariance Lemma 2: Denote by Z1;:::;ZN , N = m + n, the matrix estimator for the vector θ^ in (4), as concatenated sequence of X1;:::;Xm and Y1;:::;Yn. Then 1 1 n S = S 10 + S 01: (9) (Xi Yj) = TZ (Xi) TX (Xi) (14) m n H − − j=1 X When the vector θ^ contains only one element, that is, r = s = m 1 in (7) and (8), the covariance estimator in (9) reduces to a (Yj Xi) = TZ (Yj) TY (Yj) (15) H − − variance estimator (θ^). i=1 V X Proof: It follows from (12) along with the definition of III. LINEARITHMIC ALGORITHMS the Z-sequence that Possessing the time complexities of orders (mn) and N 2 O ^ 1 [kmn + k (m + n)], respectively, the algorithm of θ in TZ (Xi) = (Xi Zj) + O H − 2 (1) and the algorithm of S in (9) are both computationally j=1 X inefficient, especially when the sample sizes m and n are large. m 1 n However, by the relationship of the Heaviside function and = (Xi Xj) + + (Xi Yj) H − 2 H − j=1 j=1 mid-ranks shown in Lemma 1 below, linearithmic algorithms X X can be formulated based on DeLong’s formulas (5)–(9). TX (Xi) | {z } SUNSUNetet al. al.: FAST: FAST IMPLEMENTATION IMPLEMENTATION OF OF DELONG’S DELONG’S ALGORITHM ALGORITHM FOR COMPARING COMPARING THE THE AREAS AREAS UNDER UNDER CORRELATED CORRELATED ROC ROC CURVES CURVES 3 3 C.
Load more

Recommended publications
  • Krishnan Correlation CUSB.Pdf
    SELF INSTRUCTIONAL STUDY MATERIAL PROGRAMME: M.A. DEVELOPMENT STUDIES COURSE: STATISTICAL METHODS FOR DEVELOPMENT RESEARCH (MADVS2005C04) UNIT III-PART A CORRELATION PROF.(Dr). KRISHNAN CHALIL 25, APRIL 2020 1 U N I T – 3 :C O R R E L A T I O N After reading this material, the learners are expected to : . Understand the meaning of correlation and regression . Learn the different types of correlation . Understand various measures of correlation., and, . Explain the regression line and equation 3.1. Introduction By now we have a clear idea about the behavior of single variables using different measures of Central tendency and dispersion. Here the data concerned with one variable is called ‗univariate data‘ and this type of analysis is called ‗univariate analysis‘. But, in nature, some variables are related. For example, there exists some relationships between height of father and height of son, price of a commodity and amount demanded, the yield of a plant and manure added, cost of living and wages etc. This is a a case of ‗bivariate data‘ and such analysis is called as ‗bivariate data analysis. Correlation is one type of bivariate statistics. Correlation is the relationship between two variables in which the changes in the values of one variable are followed by changes in the values of the other variable. 3.2. Some Definitions 1. ―When the relationship is of a quantitative nature, the approximate statistical tool for discovering and measuring the relationship and expressing it in a brief formula is known as correlation‖—(Craxton and Cowden) 2. ‗correlation is an analysis of the co-variation between two or more variables‖—(A.M Tuttle) 3.
    [Show full text]
  • ED441031.Pdf
    DOCUMENT RESUME ED 441 031 TM 030 830 AUTHOR Fahoome, Gail; Sawilowsky, Shlomo S. TITLE Review of Twenty Nonparametric Statistics and Their Large Sample Approximations. PUB DATE 2000-04-00 NOTE 42p.; Paper presented at the Annual Meeting of the American Educational Research Association (New Orleans, LA, April 24-28, 2000). PUB TYPE Information Analyses (070)-- Numerical/Quantitative Data (110) Speeches/Meeting Papers (150) EDRS PRICE MF01/PCO2 Plus Postage. DESCRIPTORS Monte Carlo Methods; *Nonparametric Statistics; *Sample Size; *Statistical Distributions ABSTRACT Nonparametric procedures are often more powerful than classical tests for real world data, which are rarely normally distributed. However, there are difficulties in using these tests. Computational formulas are scattered tnrous-hout the literature, and there is a lack of avalialalicy of tables of critical values. This paper brings together the computational formulas for 20 commonly used nonparametric tests that have large-sample approximations for the critical value. Because there is no generally agreed upon lower limit for the sample size, Monte Carlo methods have been used to determine the smallest sample size that can be used with the large-sample approximations. The statistics reviewed include single-population tests, comparisons of two populations, comparisons of several populations, and tests of association. (Contains 4 tables and 59 references.)(Author/SLD) Reproductions supplied by EDRS are the best that can be made from the original document. Review of Twenty Nonparametric Statistics and Their Large Sample Approximations Gail Fahoome Shlomo S. Sawilowsky U.S. DEPARTMENT OF EDUCATION Mice of Educational Research and Improvement PERMISSION TO REPRODUCE AND UCATIONAL RESOURCES INFORMATION DISSEMINATE THIS MATERIAL HAS f BEEN GRANTED BY CENTER (ERIC) This document has been reproduced as received from the person or organization originating it.
    [Show full text]
  • Detecting Trends Using Spearman's Rank Correlation Coefficient
    Environmental Forensics 2001) 2, 359±362 doi:10.1006/enfo.2001.0061, available online at http://www.idealibrary.com on Detecting Trends Using Spearman's Rank Correlation Coecient Thomas D. Gauthier Sciences International Inc, 6150 State Road 70, East Bradenton, FL 34203, U.S.A. Received 23 August 2001, Revised manuscript accepted 1 October 2001) Spearman's rank correlation coecient is a useful tool for exploratory data analysis in environmental forensic investigations. In this application it is used to detect monotonic trends in chemical concentration with time or space. # 2001 AEHS Keywords: Spearman rank correlation coecient; trend analysis; soil; groundwater. Environmental forensic investigations can involve a The idea behind the rank correlation coecient is variety of statistical analysis techniques. A relatively simple. Each variable is ranked separately from lowest simple technique that can be used for exploratory data to highest e.g. 1, 2, 3, etc.) and the dierence between analysis is the Spearman rank correlation coecient. In ranks for each data pair is recorded. If the data are this technical note, I describe howthe Spearman rank correlated, then the sum of the square of the dierence correlation coecient can be used as a statistical tool between ranks will be small. The magnitude of the sum to detect monotonic trends in chemical concentrations is related to the signi®cance of the correlation. with time or space. The Spearman rank correlation coecient is calcu- Trend analysis can be useful for detecting chemical lated according to the following equation: ``footprints'' at a site or evaluating the eectiveness of Pn an installed or natural attenuation remedy.
    [Show full text]
  • Correlation • References: O Kendall, M., 1938: a New Measure of Rank Correlation
    Coupling metrics to diagnose land-atmosphere interactions http://tiny.cc/l-a-metrics Correlation • References: o Kendall, M., 1938: A new measure of rank correlation. Biometrika, 30(1-2), 81–89. o Pearson, K., 1895: Notes on regression and inheritance in the case of two parents, Proc. Royal Soc. London, 58, 240–242. o Spearman, C., 1907, Demonstration of formulæ for true measurement of correlation. Amer. J. Psychol., 18(2), 161–169. • Principle: o A correlation r is a measure of how one variable tends to vary (in sync, out of sync, or randomly) with another variable in space and/or time. –1 ≤ r ≤ 1 o The most commonly used is Pearson product-moment correlation coefficient, which relates how well a distribution of two quantities fits a linear regression: cov(x, y) xy− x y r(x, y) = = 2 2 σx σy x xx y yy − − where overbars denote averages over domains in space, time or both. § As with linear regressions, there is an implied assumption that the distribution of each variable is near normal. If one or both variables are not, it may be advisable to remap them into a normal distribution. o Spearman's rank correlation coefficient relates the ranked ordering of two variables in a non-parametric fashion. This can be handy as Pearson’s r can be heavily influenced by outliers, just as linear regressions are. The equation is the same but the sorted ranks of x and y are used instead of the values of x and y. o Kendall rank correlation coefficient is a variant of rank correlation.
    [Show full text]
  • Correlation and Regression Analysis
    OIC ACCREDITATION CERTIFICATION PROGRAMME FOR OFFICIAL STATISTICS Correlation and Regression Analysis TEXTBOOK ORGANISATION OF ISLAMIC COOPERATION STATISTICAL ECONOMIC AND SOCIAL RESEARCH AND TRAINING CENTRE FOR ISLAMIC COUNTRIES OIC ACCREDITATION CERTIFICATION PROGRAMME FOR OFFICIAL STATISTICS Correlation and Regression Analysis TEXTBOOK {{Dr. Mohamed Ahmed Zaid}} ORGANISATION OF ISLAMIC COOPERATION STATISTICAL ECONOMIC AND SOCIAL RESEARCH AND TRAINING CENTRE FOR ISLAMIC COUNTRIES © 2015 The Statistical, Economic and Social Research and Training Centre for Islamic Countries (SESRIC) Kudüs Cad. No: 9, Diplomatik Site, 06450 Oran, Ankara – Turkey Telephone +90 – 312 – 468 6172 Internet www.sesric.org E-mail [email protected] The material presented in this publication is copyrighted. The authors give the permission to view, copy download, and print the material presented that these materials are not going to be reused, on whatsoever condition, for commercial purposes. For permission to reproduce or reprint any part of this publication, please send a request with complete information to the Publication Department of SESRIC. All queries on rights and licenses should be addressed to the Statistics Department, SESRIC, at the aforementioned address. DISCLAIMER: Any views or opinions presented in this document are solely those of the author(s) and do not reflect the views of SESRIC. ISBN: xxx-xxx-xxxx-xx-x Cover design by Publication Department, SESRIC. For additional information, contact Statistics Department, SESRIC. i CONTENTS Acronyms
    [Show full text]
  • "Rank and Linear Correlation Differences in Simulation and Other
    AN ABSTRACT OF THE THESIS OF Maryam Agahi for the degree of Master of Science in Industrial Engineering presented on August 9, 2013 Title: Rank and Linear Correlation Differences in Simulation and other Applications Abstract approved: David S. Kim Monte Carlo simulation is used to quantify and characterize uncertainty in a variety of applications such as financial/engineering economic analysis, and project management. The dependence or correlation between the random variables modeled can also be simulated to add more accuracy to simulations. However, there exists a difference between how correlation is most often estimated from data (linear correlation), and the correlation that is simulated (rank correlation). In this research an empirical methodology is developed to estimate the difference between the specified linear correlation between two random variables, and the resulting linear correlation when rank correlation is simulated. It is shown that in some cases there can be relatively large differences. The methodology is based on the shape of the quantile-quantile plot of two distributions, a measure of the linearity of the quantile- quantile plot, and the level of correlation between the two random variables. This methodology also gives a user the ability to estimate the rank correlation that when simulated, generates the desired linear correlation. This methodology enhances the accuracy of simulations with dependent random variables while utilizing existing simulation software tools. ©Copyright by Maryam Agahi August 9,2013 All Rights
    [Show full text]
  • A Test of Independence in Two-Way Contingency Tables Based on Maximal Correlation
    A TEST OF INDEPENDENCE IN TWO-WAY CONTINGENCY TABLES BASED ON MAXIMAL CORRELATION Deniz C. YenigÄun A Dissertation Submitted to the Graduate College of Bowling Green State University in partial ful¯llment of the requirements for the degree of DOCTOR OF PHILOSOPHY August 2007 Committee: G¶abor Sz¶ekely, Advisor Maria L. Rizzo, Co-Advisor Louisa Ha, Graduate Faculty Representative James Albert Craig L. Zirbel ii ABSTRACT G¶abor Sz¶ekely, Advisor Maximal correlation has several desirable properties as a measure of dependence, includ- ing the fact that it vanishes if and only if the variables are independent. Except for a few special cases, it is hard to evaluate maximal correlation explicitly. In this dissertation, we focus on two-dimensional contingency tables and discuss a procedure for estimating maxi- mal correlation, which we use for constructing a test of independence. For large samples, we present the asymptotic null distribution of the test statistic. For small samples or tables with sparseness, we use exact inferential methods, where we employ maximal correlation as the ordering criterion. We compare the maximal correlation test with other tests of independence by Monte Carlo simulations. When the underlying continuous variables are dependent but uncorre- lated, we point out some cases for which the new test is more powerful. iii ACKNOWLEDGEMENTS I would like to express my sincere appreciation to my advisor, G¶abor Sz¶ekely, and my co-advisor, Maria Rizzo, for their advice and help throughout this research. I thank to all the members of my committee, Craig Zirbel, Jim Albert, and Louisa Ha, for their time and advice.
    [Show full text]
  • Chi-Squared for Association in SPSS
    community project encouraging academics to share statistics support resources All stcp resources are released under a Creative Commons licence stcp-karadimitriou-chisqS The following resources are associated: Summarising categorical variables in SPSS and the ‘Titanic.sav’ dataset Chi-Squared Test for Association in SPSS Dependent variable: Categorical Independent variable: Categorical Common Applications: Association between two categorical variables. The chi-squared test tests the hypothesis that there is no relationship between two categorical variables. It compares the observed frequencies from the data with frequencies which would be expected if there was no relationship between the two variables. Data: The dataset Titanic.sav contains data on 1309 passengers and crew who were on board the ship ‘Titanic’ when it sank in 1912. The question of interest is which factors affected survival. The dependent variable is ‘Survival’ and possible independent values are all the other variables. Here we will look at whether there’s an association between nationality (Residence) and survival (Survived). Variable name pclass survived Residence Gender age sibsp parch fare No. of No. of price Class of Survived Country of Gender siblings/ parents/ Name Age of passenger 0 = died residence 0 = male spouses on children on ticket board board Abbing, Anthony 3 0 USA 0 42 0 0 7.55 Abbott, Rosa 3 1 USA 1 35 1 1 20.25 Abelseth, Karen 3 1 UK 1 16 0 0 7.65 Data of this type are usually summarised by counting the number of subjects in each factor category and presenting it in the form of a table, known as a cross-tabulation or a contingency table.
    [Show full text]
  • Linear Regression and Correlation
    NCSS Statistical Software NCSS.com Chapter 300 Linear Regression and Correlation Introduction Linear Regression refers to a group of techniques for fitting and studying the straight-line relationship between two variables. Linear regression estimates the regression coefficients β0 and β1 in the equation Yj = β0 + β1 X j + ε j where X is the independent variable, Y is the dependent variable, β0 is the Y intercept, β1 is the slope, and ε is the error. In order to calculate confidence intervals and hypothesis tests, it is assumed that the errors are independent and normally distributed with mean zero and variance σ 2 . Given a sample of N observations on X and Y, the method of least squares estimates β0 and β1 as well as various other quantities that describe the precision of the estimates and the goodness-of-fit of the straight line to the data. Since the estimated line will seldom fit the data exactly, a term for the discrepancy between the actual and fitted data values must be added. The equation then becomes y j = b0 + b1x j + e j = y j + e j 300-1 © NCSS, LLC. All Rights Reserved. NCSS Statistical Software NCSS.com Linear Regression and Correlation where j is the observation (row) number, b0 estimates β0 , b1 estimates β1 , and e j is the discrepancy between the actual data value y j and the fitted value given by the regression equation, which is often referred to as y j . This discrepancy is usually referred to as the residual. Note that the linear regression equation is a mathematical model describing the relationship between X and Y.
    [Show full text]
  • Nonparametric Statistics
    14 Nonparametric Statistics • These methods required that the data be normally CONTENTS distributed or that the sampling distributions of the 14.1 Introduction: Distribution-Free Tests relevant statistics be normally distributed. 14.2 Single-Population Inferences Where We're Going 14.3 Comparing Two Populations: • Develop the need for inferential techniques that Independent Samples require fewer or less stringent assumptions than the 14.4 Comparing Two Populations: Paired methods of Chapters 7 – 10 and 11 (14.1) Difference Experiment • Introduce nonparametric tests that are based on ranks 14.5 Comparing Three or More Populations: (i.e., on an ordering of the sample measurements Completely Randomized Design according to their relative magnitudes) (14.2–14.7) • Present a nonparametric test about the central 14.6 Comparing Three or More Populations: tendency of a single population (14.2) Randomized Block Design • Present a nonparametric test for comparing two 14.7 Rank Correlation populations with independent samples (14.3) • Present a nonparametric test for comparing two Where We've Been populations with paired samples (14.4) • Presented methods for making inferences about • Present a nonparametric test for comparing three or means ( Chapters 7 – 10 ) and for making inferences more populations using a designed experiment about the correlation between two quantitative (14.5–14.6) variables ( Chapter 11 ) • Present a nonparametric test for rank correlation (14.7) 14-1 Copyright (c) 2013 Pearson Education, Inc M14_MCCL6947_12_AIE_C14.indd 14-1 10/1/11 9:59 AM Statistics IN Action How Vulnerable Are New Hampshire Wells to Groundwater Contamination? Methyl tert- butyl ether (commonly known as MTBE) is a the measuring device, the MTBE val- volatile, flammable, colorless liquid manufactured by the chemi- ues for these wells are recorded as .2 cal reaction of methanol and isobutylene.
    [Show full text]
  • The Spearman's Rank Correlation Test
    GEOGRAPHICAL TECHNIQUES Using quantitative data Using primary data Using qualitative data Using secondary data The Spearman’s Rank Correlation Test 2 Introduction The Spearman’s rank correlation coefficient (rs) is a method of testing the strength and direction (positive or negative) of the correlation (relationship or connection) between two variables. As part of looking at Changing Places in human geography you could use data from the 2011 census online e.g. from http://ukcensusdata.com (and paper copies from local libraries for previous censuses in 2001, 1991 and 1981) to look at the relationships between pairs of data across an area such as a London borough. You can also use the site https://data.london.gov.uk/dataset/ward-profiles-and-atlas which is very easy to use. Data on gender, age, health, housing, crime, ethnicity, education and employment is easily accessed at ward level. So, if you are looking to understand and analyse a) differences between nearby places and b) how those places have changed over time, a Spearman’s Rank test could be a good way of testing the strength of your sets of data. This could help you to draw conclusions in your investigation. You could first begin by plotting your data on a scatter graph to see if there appeared to a correlation between the two sets of data. This would help you to see quickly if the relationship was a positive or negative one. You would need to do a separate Spearman’s rank correlation test for each set of pairs of data and for each time period.
    [Show full text]
  • Spearman Correlation: 2-Tailed
    Statistics Solutions Advancement Through Clarity http://www.statisticssolutions.com Spearman Correlation: 2-tailed Large Effect Size Sample size for a Spearman correlation was determined using power analysis. The power analysis was conducted in G-POWER using an alpha of 0.05, a power of 0.80, and a large effect size (? = 0.5) for a two-tailed test. Because Spearman’s rank correlation coefficient is computationally identical to Pearson product-moment coefficient, power analysis was conducted using software for estimating power of a Pearson’s correlation. Based on the aforementioned assumptions, the required sample size was determined to be 29. Medium Effect Size Sample size for a Spearman correlation was determined using power analysis. The power analysis was conducted in G-POWER using an alpha of 0.05, a power of 0.80, and a medium effect size (? = 0.3) for a two-tailed test. Because Spearman’s rank correlation coefficient is computationally identical to Pearson product-moment coefficient, power analysis was conducted using software for estimating power of a Pearson’s correlation. Based on the aforementioned assumptions, the required sample size was determined to be 82. Small Effect Size Sample size for a Spearman correlation was determined using power analysis. The power analysis was conducted in G-POWER using an alpha of 0.05, a power of 0.80, and a small effect size (? = 0.1) for a two-tailed test. Because Spearman’s rank correlation coefficient is computationally identical to Pearson product-moment coefficient, power analysis was conducted using software for estimating power of a Pearson’s correlation.
    [Show full text]