DOCSLIB.ORG

Outliers Outliers Are Data Points Which Lie Outside the General Linear Pattern of Which the Midline Is the Regression Line

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Outliers Outliers Are Data Points Which Lie Outside the General Linear Pattern of Which the Midline Is the Regression Line Outliers Outliers are data points which lie outside the general linear pattern of which the midline is the regression line. A rule of thumb is that outliers are points whose standardized residual is greater than 3.3 (corresponding to the .001 alpha level). The removal of outliers from the data set under analysis can at times dramatically affect the performance of a regression model. Outliers should be removed if there is reason to believe that other variables not in the model explain why the outlier cases are unusual -- that is, these cases need a separate model. Alternatively, outliers may suggest that additional explanatory variables need to be brought into the model (that is, the model needs respecification). Another alternative is to use robust regression, whose algorithm gives less weight to outliers but does not discard them. The leverage statistic, h, also called the hat-value, is available to identify cases which influence the regression model more than others. • Belsley, Kuh, and Welsch (1980) define the leverage (hi ) of the ith observation as & 2 1 (xi x¯) hi = % n & 2 (n 1)Sx Leverage assesses how far away a value of the independent variable value is from the mean value: the farther away the observation the more leverage it has. From the definition you can see that leverage is mitigated by a larger sample size (any single point should have less influence) and by a larger variance of the independent variable (again, any single point should have less influence). • 0<h<1 The leverage statistic varies from 0 (no influence on the model) to 1 (completely determines the model). • A rule of thumb is that cases with leverage under .2 are not a problem, but if a case has leverage over .5, the case has undue leverage and should be examined for the possibility of measurement error or the need to model such cases separately. STATA command predict h, hat. Cook's distance, D, is another measure of the influence of a case. Cook's distance measures the effect of deleting a given observation. • Observations with larger D values than the rest of the data are those which have unusual leverage. • D > 4/n the criterion to indicate a possible problem. STATA command predict D, cooksd dfbetas, is another statistic for assessing the influence of a case. • If dfbetas > 0, the case increases the slope; if <0, the case decreases the slope. • The case may be considered an influential outlier if |dfbetas| > 2/%n. STATA command dfbeta creates dfbeta’s for all variables. or predict DFx1, dfbeta(x1) for individual variables dfFit. DfFit measues how much the estimate changes as a result of a particular observation being dropped from analysis. dfits is defined as h • dfits = Rstudent i . Where Rstudent is the studentized residual. i & 1 hi • The case may be considered an influential outlier if DFITS> 2/%k/n STATA command predict DFITS, dfits Studentized residuals and deleted studentized residuals are also used to detect outliers with high leverage. A "studentized residual" is the observed residual divided by the standard deviation. The "studentized deleted residual," also called the "jacknife residual," is the observed residual divided by the standard deviation computed with the given observation left out of the analysis. Analysis of outliers usually focuses on deleted residuals. Other synonyms include externally studentized residual or, misleadingly, standardized residual. There will be a t value for each residual, with df - n - k - 1, where k is the number of independent variables. When t exceeds the critical value for a given alpha level (ex., .05) then the case is considered an outlier. In a plot of deleted studentized residuals versus ordinary residuals, one may draw lines at plus and minus two standard units to highlight cases outside the range where 95% of the cases normally lie; points substantially off the straight line are potential leverage problems. STATA command predict student, rstudent or predict standard, rstandard Partial regression plots, also called partial regression leverage plots or added variable plots, are yet another way of detecting influential sets of cases. Partial regression plots are a series of bivariate regression plots of the dependent variable with each of the independent variables in turn. The plots show cases by number or label instead of dots. One looks for cases which are outliers on all or many of the plots. STATA command avplots Example using the Murder.dta data set. reg mrdrte exec unem d90 d93 Source SS df MS Number of obs= 153 F( 4, 148) = 3.05 Model 977.390644 4 244.347661 Prob > F = 0.0190 Residual 11867.9475 148 80.1888343 R-squared = 0.0761 Adj R-squared= 0.0511 Total 12845.3381 152 84.5088034 Root MSE = 8.9548 mrdrte Coef. Std. Err. t P>t [95% Conf. Interval] exec .1627547 .1939295 0.84 0.403 -.2204738 .5459832 unem 1.390786 .4508653 3.08 0.002 .4998207 2.281751 d90 2.675335 1.816934 1.47 0.143 -.91515 6.26582 d93 1.607317 1.774768 0.91 0.367 -1.899842 5.114476 _cons -1.864393 3.069517 -0.61 0.545 -7.930134 4.201349 predict e, residual predict yhat predict standard, rstandard predict student, rstudent predict h, hat predict D, cooksd predict DFITS, dfits predict W, welsch dfbeta DFexec: DFbeta(exec) DFunem: DFbeta(unem) DFd90: DFbeta(d90) DFd93: DFbeta(d93) rvplot 60 40 20 Residuals 0 -20 0 5 10 15 Fitted values To create a standardized residual plot graph twoway scatter standard yhat, yline(0) 8 6 4 2 Standardized residuals 0 -2 0 5 10 15 Fitted values To identify the outliers. graph twoway scatter standard yhat, yline(0) mlabel(state) 8 DC DC 6 4 DC 2 LA Standardized residuals NY NY LA MD CAFLNCMDGA MDMS DE NCVA SCNC SCTNTNGACAGA MI NY TX NH MACT NEHINJ AZMONVINVATN NVNVAR MIILMOILFLAL ALMS 0 VTRI NE SDHINEKS PA KSORCOIN AZPAOKILVAOKKYAZALMONM AKSCNMMIAR CA SDME IAUTHIMNWIWIKSWIDEWANJOHDECTINWYCTKYWAOHOHCO ARPA NMKYAKFL MSAKTX NDNDIASD IAMNUTUTVTWYMNCOMTORMTMT NJ TX ND VTMEIDNHIDNHMAIDMAORRIWARI WV LA WYME WV WV -2 0 5 10 15 Fitted values avplots, mlabel(state) DCDC 80 60 DC DC 60 40 DC 40 20 DC LA LA MSNYMDNC FL GATX 20 e( mrdrte | X ) NYMIMDTNCATNMDNCCTDESCNCSCGAGAVAMO TX e( mrdrte | X ) 0 NVTNNHARNVVA FL VA TX KYNMAKPAARILMIOHCTKYOKILMIOHAKDECOORNMKSINMOILAZWIKYOHPAMNSCHIINIADECTNJWANEAZKSWIKSMESDMSCANJHIRIINVTNEHINVMAUTNEOKALAROKAZ ALMOFL LA AKRIWVNJORIDMACOWAIDMTRIMAVTMNWIIDNHMTWAORNDIAWYMEVTNDCOMNIANDWYSDUTUT LA NY LA WVMEWYNH MDNCNCFLNYGANCSCMDSCGATNCAMOTNARGANVMDNVMOMSTXCATNMIILNMALILSCALARALNYMIMIMSNMCA MS AK LA 0 NVAZFL OKAKAR KY DECTHINJVAVAKSVAKSCOINPANJOKAZINAZKYCTORPAMOINOKOHOHFLKYPAOHTXMTNMWACOAK WV WV NHNENEMASDVTHIRIUTIASDHINDMESDWINDNETXIAUTMNNDKSWIDEWYVTMNIAMNWACOVTMEDECTMTIDWIWYUTORNHNHIDMTMAORNJWARIRIMEID WY -20 0 10 20 30 -4 -2 0 2 4 6 e( exec | X ) e( unem | X ) coef = .16275467, se = .19392954, t = .84 coef = 1.3907856, se = .45086525, t = 3.08 80 DC DC DC DC 60 60 40 40 DC DC 20 20 LA LA NYLA MDNY LAMDNY NCMDMS e( mrdrte | X ) MS NCMD e( mrdrte | X ) GAFLVASCNCCA TXGASCTNNCCA MOARGAILTNNVNY MDNYFLGACA NCTNGAMDNY VASCNVTNGACATX MS 0 NVNHHINJDECTAZMOTNMI FLALVANVMSILMI VAAZOKALCASCNEKSMIIN 0 DEVANCSC MI ARNVMOILAL CA AZFLNMILARALMI VTKSALMARIPAOKNMIL MOARNEOKPAKYHIAZNM FLUTDECTPANMSDHIWICOKYAK NHMACTVTRIHINJSDMENEKSMNPAWIORINAZNVMOTNOHOKILWACOALARTXKYNMMSAKNESDUTIAHINDWIVAKSMNDECOINWYVTOKCTKYAZOHTXFLPAMINJNMSCAK NEHISDNDIAUTKSWIMNCOWAVTMECTDENJINWYORPAMTOHOKMOKYMARIAK TXLAUTNDIAMNMSINSDMENEMTWIWACOKYAKOROHAR UTIDNDIAMNVTMEORMARISDMTWIKSWACONJCTDEINWYOHAK TX WAWYNHRIMTIDMAORNJIANDMNVTOH NDIAUTMTIDWY WVLA MTIDNHMAORWAMERI WV NHID WV WYWVID NHWV MEWV -20 -20 -1 -.5 0 .5 -.5 0 .5 e( d90 | X ) e( d93 | X ) coef = 2.6753348, se = 1.8169343, t = 1.47 coef = 1.6073174, se = 1.774768, t = .91 To graphically measure the influence of observations graph twoway scatter standard yhat [aweight=D], msymbol(oh) yline(0) 8 6 4 2 Standardized residuals 0 -2 0 5 10 15 Fitted values Leverage versus residual squared plot marks the means of leverage and squared residuals. Leverage tells us how much potential for influencing the regression an observation has. lvr2plot To examine the numerical measure for outliers. leverage h>2k/n Cook’s D>4/n DFITS>2/%k/n Welsch’s W>3/%k DFBETA>2/%n For example, sort D list state yhat D DFITS W in -5/l state yhat D DFITS W 149. WV 13.15609 .0150447 -.2742131 -3.513339 150. DC 6.897557 .0452811 .4927538 6.137678 151. TX 15.01208 .0504226 -.500824 -8.794935 152. DC 9.990127 .2905912 1.547051 19.3077 153. DC 11.5646 .4116747 1.832155 22.9866 Recall that D > 4/n indicates a possible problem. With n=153, D > .02614379 DFITS> 2/%k/n may be considered an influential outlier. With n=153, k=4, DFITS>12.369317.
Load more

Recommended publications
  • A Review on Outlier/Anomaly Detection in Time Series Data
    A review on outlier/anomaly detection in time series data ANE BLÁZQUEZ-GARCÍA and ANGEL CONDE, Ikerlan Technology Research Centre, Basque Research and Technology Alliance (BRTA), Spain USUE MORI, Intelligent Systems Group (ISG), Department of Computer Science and Artificial Intelligence, University of the Basque Country (UPV/EHU), Spain JOSE A. LOZANO, Intelligent Systems Group (ISG), Department of Computer Science and Artificial Intelligence, University of the Basque Country (UPV/EHU), Spain and Basque Center for Applied Mathematics (BCAM), Spain Recent advances in technology have brought major breakthroughs in data collection, enabling a large amount of data to be gathered over time and thus generating time series. Mining this data has become an important task for researchers and practitioners in the past few years, including the detection of outliers or anomalies that may represent errors or events of interest. This review aims to provide a structured and comprehensive state-of-the-art on outlier detection techniques in the context of time series. To this end, a taxonomy is presented based on the main aspects that characterize an outlier detection technique. Additional Key Words and Phrases: Outlier detection, anomaly detection, time series, data mining, taxonomy, software 1 INTRODUCTION Recent advances in technology allow us to collect a large amount of data over time in diverse research areas. Observations that have been recorded in an orderly fashion and which are correlated in time constitute a time series. Time series data mining aims to extract all meaningful knowledge from this data, and several mining tasks (e.g., classification, clustering, forecasting, and outlier detection) have been considered in the literature [Esling and Agon 2012; Fu 2011; Ratanamahatana et al.
    [Show full text]
  • Outlier Identification.Pdf
    STATGRAPHICS – Rev. 7/6/2009 Outlier Identification Summary The Outlier Identification procedure is designed to help determine whether or not a sample of n numeric observations contains outliers. By “outlier”, we mean an observation that does not come from the same distribution as the rest of the sample. Both graphical methods and formal statistical tests are included. The procedure will also save a column back to the datasheet identifying the outliers in a form that can be used in the Select field on other data input dialog boxes. Sample StatFolio: outlier.sgp Sample Data: The file bodytemp.sgd file contains data describing the body temperature of a sample of n = 130 people. It was obtained from the Journal of Statistical Education Data Archive (www.amstat.org/publications/jse/jse_data_archive.html) and originally appeared in the Journal of the American Medical Association. The first 20 rows of the file are shown below. Temperature Gender Heart Rate 98.4 Male 84 98.4 Male 82 98.2 Female 65 97.8 Female 71 98 Male 78 97.9 Male 72 99 Female 79 98.5 Male 68 98.8 Female 64 98 Male 67 97.4 Male 78 98.8 Male 78 99.5 Male 75 98 Female 73 100.8 Female 77 97.1 Male 75 98 Male 71 98.7 Female 72 98.9 Male 80 99 Male 75 2009 by StatPoint Technologies, Inc. Outlier Identification - 1 STATGRAPHICS – Rev. 7/6/2009 Data Input The data to be analyzed consist of a single numeric column containing n = 2 or more observations.
    [Show full text]
  • A Machine Learning Approach to Outlier Detection and Imputation of Missing Data1
    Ninth IFC Conference on “Are post-crisis statistical initiatives completed?” Basel, 30-31 August 2018 A machine learning approach to outlier detection and imputation of missing data1 Nicola Benatti, European Central Bank 1 This paper was prepared for the meeting. The views expressed are those of the authors and do not necessarily reflect the views of the BIS, the IFC or the central banks and other institutions represented at the meeting. A machine learning approach to outlier detection and imputation of missing data Nicola Benatti In the era of ready-to-go analysis of high-dimensional datasets, data quality is essential for economists to guarantee robust results. Traditional techniques for outlier detection tend to exclude the tails of distributions and ignore the data generation processes of specific datasets. At the same time, multiple imputation of missing values is traditionally an iterative process based on linear estimations, implying the use of simplified data generation models. In this paper I propose the use of common machine learning algorithms (i.e. boosted trees, cross validation and cluster analysis) to determine the data generation models of a firm-level dataset in order to detect outliers and impute missing values. Keywords: machine learning, outlier detection, imputation, firm data JEL classification: C81, C55, C53, D22 Contents A machine learning approach to outlier detection and imputation of missing data ... 1 Introduction ..............................................................................................................................................
    [Show full text]
  • Chapter 4: Model Adequacy Checking
    Chapter 4: Model Adequacy Checking In this chapter, we discuss some introductory aspect of model adequacy checking, including: • Residual Analysis, • Residual plots, • Detection and treatment of outliers, • The PRESS statistic • Testing for lack of fit. The major assumptions that we have made in regression analysis are: • The relationship between the response Y and the regressors is linear, at least approximately. • The error term ε has zero mean. • The error term ε has constant varianceσ 2 . • The errors are uncorrelated. • The errors are normally distributed. Assumptions 4 and 5 together imply that the errors are independent. Recall that assumption 5 is required for hypothesis testing and interval estimation. Residual Analysis: The residuals , , , have the following important properties: e1 e2 L en (a) The mean of is 0. ei (b) The estimate of population variance computed from the n residuals is: n n 2 2 ∑()ei−e ∑ei ) 2 = i=1 = i=1 = SS Re s = σ n − p n − p n − p MS Re s (c) Since the sum of is zero, they are not independent. However, if the number of ei residuals ( n ) is large relative to the number of parameters ( p ), the dependency effect can be ignored in an analysis of residuals. Standardized Residual: The quantity = ei ,i = 1,2, , n , is called d i L MS Re s standardized residual. The standardized residuals have mean zero and approximately unit variance. A large standardized residual ( > 3 ) potentially indicates an outlier. d i Recall that e = (I − H )Y = (I − H )(X β + ε )= (I − H )ε Therefore, / Var()e = var[]()I − H ε = (I − H )var(ε )(I −H ) = σ 2 ()I − H .
    [Show full text]
  • Outlier Detection and Influence Diagnostics in Network Meta- Analysis
    Outlier detection and influence diagnostics in network meta- analysis Hisashi Noma, PhD* Department of Data Science, The Institute of Statistical Mathematics, Tokyo, Japan ORCID: http://orcid.org/0000-0002-2520-9949 Masahiko Gosho, PhD Department of Biostatistics, Faculty of Medicine, University of Tsukuba, Tsukuba, Japan Ryota Ishii, MS Biostatistics Unit, Clinical and Translational Research Center, Keio University Hospital, Tokyo, Japan Koji Oba, PhD Interfaculty Initiative in Information Studies, Graduate School of Interdisciplinary Information Studies, The University of Tokyo, Tokyo, Japan Toshi A. Furukawa, MD, PhD Departments of Health Promotion and Human Behavior, Kyoto University Graduate School of Medicine/School of Public Health, Kyoto, Japan *Corresponding author: Hisashi Noma Department of Data Science, The Institute of Statistical Mathematics 10-3 Midori-cho, Tachikawa, Tokyo 190-8562, Japan TEL: +81-50-5533-8440 e-mail: [email protected] Abstract Network meta-analysis has been gaining prominence as an evidence synthesis method that enables the comprehensive synthesis and simultaneous comparison of multiple treatments. In many network meta-analyses, some of the constituent studies may have markedly different characteristics from the others, and may be influential enough to change the overall results. The inclusion of these “outlying” studies might lead to biases, yielding misleading results. In this article, we propose effective methods for detecting outlying and influential studies in a frequentist framework. In particular, we propose suitable influence measures for network meta-analysis models that involve missing outcomes and adjust the degree of freedoms appropriately. We propose three influential measures by a leave-one-trial-out cross-validation scheme: (1) comparison-specific studentized residual, (2) relative change measure for covariance matrix of the comparative effectiveness parameters, (3) relative change measure for heterogeneity covariance matrix.
    [Show full text]
  • Residuals, Part II
    Biostatistics 650 Mon, November 5 2001 Residuals, Part II Key terms External Studentization Outliers Added Variable Plot — Partial Regression Plot Partial Residual Plot — Component Plus Residual Plot Key ideas/results ¢ 1. An external estimate of ¡ comes from refitting the model without observation . Amazingly, it has an easy formula: £ ¦!¦ ¦ £¥¤§¦©¨ "# 2. Externally Studentized Residuals $ ¦ ¦©¨ ¦!¦)( £¥¤§¦&% ' Ordinary residuals standardized with £*¤§¦ . Also known as R-Student. 3. Residual Taxonomy Names Definition Distribution ¦!¦ ¦+¨-,¥¦ ,0¦ 1 Ordinary ¡ 5 /. &243 ¦768¤§¦©¨ ¦ ¦!¦ 1 ¦!¦ PRESS ¡ &243 9' 5 $ Studentized ¦©¨ ¦ £ ¦!¦ ¤0> % = Internally Studentized : ;<; $ $ Externally Studentized ¦+¨ ¦ £?¤§¦ ¦!¦ ¤0>¤A@ % R-Student = 4. Outliers are unusually large observations, due to an unmodeled shift or an (unmodeled) increase in variance. 5. Outliers are not necessarily bad points; they simply are not consistent with your model. They may posses valuable information about the inadequacies of your model. 1 PRESS Residuals & Studentized Residuals Recall that the PRESS residual has a easy computation form ¦ ¦©¨ PRESS ¦!¦ ( ¦!¦ It’s easy to show that this has variance ¡ , and hence a standardized PRESS residual is 9 ¦ ¦ ¦!¦ ¦ PRESS ¨ ¨ ¨ ¦ : £ ¦!¦ £ ¦!¦ £ ¦ ¦ % % % ' When we standardize a PRESS residual we get the studentized residual! This is very informa- ,¥¦ ¦ tive. We understand the PRESS residual to be the residual at ¦ if we had omitted from the 3 model. However, after adjusting for it’s variance, we get the same thing as a studentized residual. Hence the standardized residual can be interpreted as a standardized PRESS residual. Internal vs External Studentization ,*¦ ¦ The PRESS residuals remove the impact of point ¦ on the fit at . But the studentized 3 ¢ ¦ ¨ ¦ £?% ¦!¦ £ residual : can be corrupted by point by way of ; a large outlier will inflate the residual mean square, and hence £ .
    [Show full text]
  • Chapter 39 Fit Analyses
    Chapter 39 Fit Analyses Chapter Table of Contents STATISTICAL MODELS ............................568 LINEAR MODELS ...............................569 GENERALIZED LINEAR MODELS .....................572 The Exponential Family of Distributions ....................572 LinkFunction..................................573 The Likelihood Function and Maximum-Likelihood Estimation . .....574 ScaleParameter.................................576 Goodness of Fit .................................576 Quasi-Likelihood Functions ..........................577 NONPARAMETRIC SMOOTHERS ......................580 SmootherDegreesofFreedom.........................581 SmootherGeneralizedCrossValidation....................582 VARIABLES ...................................583 METHOD .....................................585 OUTPUT .....................................587 TABLES ......................................590 ModelInformation...............................590 ModelEquation.................................590 X’X Matrix . .................................591 SummaryofFitforLinearModels.......................592 SummaryofFitforGeneralizedLinearModels................593 AnalysisofVarianceforLinearModels....................594 AnalysisofDevianceforGeneralizedLinearModels.............595 TypeITests...................................595 TypeIIITests..................................597 ParameterEstimatesforLinearModels....................599 ParameterEstimatesforGeneralizedLinearModels..............601 C.I.forParameters...............................602 Collinearity
    [Show full text]
  • Effects of Outliers with an Outlier
    • The mean is a good measure to use to describe data that are close in value. • The median more accurately describes data Effects of Outliers with an outlier. • The mode is a good measure to use when you have categorical data; for example, if each student records his or her favorite color, the color (a category) listed most often is the mode of the data. In the data set below, the value 12 is much less than Additional Example 2: Exploring the Effects of the other values in the set. An extreme value such as Outliers on Measures of Central Tendency this is called an outlier. The data shows Sara’s scores for the last 5 35, 38, 27, 12, 30, 41, 31, 35 math tests: 88, 90, 55, 94, and 89. Identify the outlier in the data set. Then determine how the outlier affects the mean, median, and x mode of the data. x x xx x x x 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 55, 88, 89, 90, 94 outlier 55 1 Additional Example 2 Continued Additional Example 2 Continued With the Outlier Without the Outlier 55, 88, 89, 90, 94 55, 88, 89, 90, 94 outlier 55 mean: median: mode: mean: median: mode: 55+88+89+90+94 = 416 55, 88, 89, 90, 94 88+89+90+94 = 361 88, 89,+ 90, 94 416 5 = 83.2 361 4 = 90.25 2 = 89.5 The mean is 83.2. The median is 89. There is no mode.
    [Show full text]
  • How Significant Is a Boxplot Outlier?
    Journal of Statistics Education, Volume 19, Number 2(2011) How Significant Is A Boxplot Outlier? Robert Dawson Saint Mary’s University Journal of Statistics Education Volume 19, Number 2(2011), www.amstat.org/publications/jse/v19n2/dawson.pdf Copyright © 2011 by Robert Dawson all rights reserved. This text may be freely shared among individuals, but it may not be republished in any medium without express written consent from the author and advance notification of the editor. Key Words: Boxplot; Outlier; Significance; Spreadsheet; Simulation. Abstract It is common to consider Tukey’s schematic (“full”) boxplot as an informal test for the existence of outliers. While the procedure is useful, it should be used with caution, as at least 30% of samples from a normally-distributed population of any size will be flagged as containing an outlier, while for small samples (N<10) even extreme outliers indicate little. This fact is most easily seen using a simulation, which ideally students should perform for themselves. The majority of students who learn about boxplots are not familiar with the tools (such as R) that upper-level students might use for such a simulation. This article shows how, with appropriate guidance, a class can use a spreadsheet such as Excel to explore this issue. 1. Introduction Exploratory data analysis suddenly got a lot cooler on February 4, 2009, when a boxplot was featured in Randall Munroe's popular Webcomic xkcd. 1 Journal of Statistics Education, Volume 19, Number 2(2011) Figure 1 “Boyfriend” (Feb. 4 2009, http://xkcd.com/539/) Used by permission of Randall Munroe But is Megan justified in claiming “statistical significance”? We shall explore this intriguing question using Microsoft Excel.
    [Show full text]
  • Lecture 14 Testing for Kurtosis
    9/8/2016 CHE384, From Data to Decisions: Measurement, Kurtosis Uncertainty, Analysis, and Modeling • For any distribution, the kurtosis (sometimes Lecture 14 called the excess kurtosis) is defined as Testing for Kurtosis 3 (old notation = ) • For a unimodal, symmetric distribution, Chris A. Mack – a positive kurtosis means “heavy tails” and a more Adjunct Associate Professor peaked center compared to a normal distribution – a negative kurtosis means “light tails” and a more spread center compared to a normal distribution http://www.lithoguru.com/scientist/statistics/ © Chris Mack, 2016Data to Decisions 1 © Chris Mack, 2016Data to Decisions 2 Kurtosis Examples One Impact of Excess Kurtosis • For the Student’s t • For a normal distribution, the sample distribution, the variance will have an expected value of s2, excess kurtosis is and a variance of 6 2 4 1 for DF > 4 ( for DF ≤ 4 the kurtosis is infinite) • For a distribution with excess kurtosis • For a uniform 2 1 1 distribution, 1 2 © Chris Mack, 2016Data to Decisions 3 © Chris Mack, 2016Data to Decisions 4 Sample Kurtosis Sample Kurtosis • For a sample of size n, the sample kurtosis is • An unbiased estimator of the sample excess 1 kurtosis is ∑ ̅ 1 3 3 1 6 1 2 3 ∑ ̅ Standard Error: • For large n, the sampling distribution of 1 24 2 1 approaches Normal with mean 0 and variance 2 1 of 24/n 3 5 • For small samples, this estimator is biased D. N. Joanes and C. A. Gill, “Comparing Measures of Sample Skewness and Kurtosis”, The Statistician, 47(1),183–189 (1998).
    [Show full text]
  • Linear Regression Diagnostics*
    LIBRARY OF THE MASSACHUSETTS INSTITUTE OF TECHNOLOGY WORKING PAPER ALFRED P. SLOAN SCHOOL OF MANAGEMENT LINEAR REGRESSION DIAGNOSTICS* Soy E. Welsch and Edwin Kuh Massachusetts Institute of Technology and NBER Computer Research Center WP 923-77 April 1977 MASSACHUSETTS INSTITUTE OF TECHNOLOGY 50 MEMORIAL DRIVE CAMBRIDGE, MASSACHUSETTS 02139 ^ou to divest? access to expanded sub-regional markets possibility of royalty payments to parent access to local long term capital advantages given to a SAICA (i.e., reduce LINEAR REGRESSION DIAGNOSTICS^ Roy E. Welsch and Edwin Kuh Massachusetts Institute of Technology and NBER Computer Research Center WP 923-77 April 1977 *Me are indebted to the National Science Foundation for supporting this research under Grant SOC76-143II to the NBER Computer Research Center. - , ABSTFACT This paper atxeT.pts to provide the user of linear multiple regression with a batter;/ of iiagncstic tools to determine which, if any, data points have high leverage or irifluerice on the estimation process and how these pcssidly iiscrepar.t iata points -differ from the patterns set by the majority of the data. Tr.e point of viev; taken is that when diagnostics indicate the presence of anomolous data, the choice is open as to whether these data are in fact -unus-ual and helprul, or possioly hannful and thus in need of modifica- tions or deletion. The methodology/ developed depends on differences, derivatives, and decompositions of basic re:gressicn statistics. Th.ere is also a discussion of hov; these tecr-niques can be used with robust and ridge estimators, r^i exarripls is given showing the use of diagnostic methods in the estimation of a cross cou.ntr>' savir.gs rate model.
    [Show full text]
  • Studentized Residuals
    CHECKING ASSUMPTIONS/CONDITIONS AND FIXING PROBLEMS Recall, linear model is: YXiii01 where we assume: E{ε} = 0, Variance of the “errors” = σ2{ε} = σ2, same value at any X Sometimes assume errors are normally distributed (“normal errors” model). ˆ After we fit the model using data, the residuals eYYiii will be useful for checking the assumptions and conditions. But what is a large residual? If discussing GPA, 0.5 could be large but if discussion SAT scores (possible scores of 200 to 800), 0.5 would be tiny. Need to create equivalent of a z-score. Recall estimate of σ is sMSE * e i e i “Semi-studentized” residuals (p. 103) MSE Note these are called standardized residuals in R. Later, will learn another version, which R calls studentized residuals. Assumptions in the Normal Linear Regression Model A1: There is a linear relationship between X and Y. A2: The error terms (and thus the Y’s at each X) have constant variance. A3: The error terms are independent. A4: The error terms (and thus the Y’s at each X) are normally distributed. Note: In practice, we are looking for a fairly symmetric distribution with no major outliers. Other things to check (Questions to ask): Q5: Are there any major outliers in the data (X, or combination of (X,Y))? Q6: Are there other possible predictors that should be included in the model? Applet for illustrating the effect of outliers on the regression line and correlation: http://illuminations.nctm.org/LessonDetail.aspx?ID=L455 Useful Plots for Checking Assumptions and Answering These Questions Reminders: ˆ Residual
    [Show full text]