DOCSLIB.ORG

Local Regression Models: Advancements

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Local Regression Models: Advancements LOCAL REGRESSION MODELS: ADVANCEMENTS, APPLICATIONS, AND NEW METHODS A Dissertation Submitted to the Faculty of Purdue University by Ryan P. Hafen In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy May 2010 Purdue University West Lafayette, Indiana ii [Put dedication here.] iii ACKNOWLEDGMENTS [Put statement of appreciation or recognition of special assistance here.] iv TABLE OF CONTENTS Page LIST OF TABLES :::::::::::::::::::::::::::::::: vii LIST OF FIGURES ::::::::::::::::::::::::::::::: viii SYMBOLS :::::::::::::::::::::::::::::::::::: xv ABBREVIATIONS :::::::::::::::::::::::::::::::: xix ABSTRACT ::::::::::::::::::::::::::::::::::: xx 1 Background :::::::::::::::::::::::::::::::::: 1 1.1 Regression Models ::::::::::::::::::::::::::: 1 1.2 Local Regression Models :::::::::::::::::::::::: 1 1.2.1 Fitting a Local Regression Model ::::::::::::::: 2 1.2.2 Loess as a Linear Operator ::::::::::::::::::: 4 1.2.3 Statistical Properties :::::::::::::::::::::: 5 1.2.4 Diagnostics and Model selection :::::::::::::::: 9 1.2.5 Computation :::::::::::::::::::::::::: 13 1.3 Time Series Models ::::::::::::::::::::::::::: 14 1.3.1 ARMA Models ::::::::::::::::::::::::: 14 1.3.2 Nonstationarity and Trends :::::::::::::::::: 15 1.3.3 Linear Filters :::::::::::::::::::::::::: 16 1.3.4 Linear Filters in the Frequency Domain :::::::::::: 17 1.4 Seasonal Trend Decomposition using Loess (STL) :::::::::: 20 1.4.1 The STL Method :::::::::::::::::::::::: 20 1.4.2 STL Smoothing Parameters :::::::::::::::::: 24 1.4.3 STL Parameter Guidelines ::::::::::::::::::: 25 1.4.4 Seasonal Parameter Selection ::::::::::::::::: 27 1.4.5 Post Smoothing ::::::::::::::::::::::::: 29 1.4.6 Computational Considerations ::::::::::::::::: 30 2 Advancements to the STL Method ::::::::::::::::::::: 31 2.1 The Operator Matrix of an STL Decomposition ::::::::::: 31 2.1.1 Why Calculate the Operator Matrix? ::::::::::::: 31 2.1.2 How to Calculate the Operator Matrix :::::::::::: 32 2.1.3 Variance Coefficients along the Design Space ::::::::: 36 2.1.4 Spectral Properties of Smoothing Near Endpoints :::::: 37 2.1.5 ARMA Modeling on Remainder Component ::::::::: 37 2.1.6 Predicting Ahead :::::::::::::::::::::::: 39 v Page 2.1.7 Confidence and Prediction Intervals :::::::::::::: 42 2.1.8 Analysis of Variance :::::::::::::::::::::: 43 2.1.9 Model Selection ::::::::::::::::::::::::: 44 2.2 Local Quadratic Support and New Parameter Selection Guidelines : 45 2.2.1 Approximating the Critical Frequency for a Loess Filter :: 45 2.2.2 Calculating q Given !, λ, and fcrit :::::::::::::: 48 2.2.3 Trend Filter Parameter Guidelines :::::::::::::: 48 2.2.4 High-Pass Filter Parameter Guidelines :::::::::::: 49 2.3 The Revision Problem ::::::::::::::::::::::::: 49 2.3.1 Reproducing Kernel Hilbert Spaces with Loess :::::::: 51 2.3.2 Blending to Lower Degree Polynomials at the Endpoints :: 52 2.3.3 Comparison of Blending and RKHS :::::::::::::: 60 2.4 Software Implementation: stl2 and operator Packages ::::::: 62 2.4.1 The stl2 Package :::::::::::::::::::::::: 63 2.4.2 The operator Package ::::::::::::::::::::: 64 2.4.3 Computational Considerations ::::::::::::::::: 65 2.4.4 Approximations ::::::::::::::::::::::::: 67 2.5 General Guidelines ::::::::::::::::::::::::::: 68 3 An Application: Syndromic Surveillance :::::::::::::::::: 71 3.1 Background ::::::::::::::::::::::::::::::: 71 3.2 Modeling the Data ::::::::::::::::::::::::::: 72 3.2.1 Model Overview ::::::::::::::::::::::::: 73 3.2.2 Model Components and Parameter Selection ::::::::: 74 3.2.3 Model Diagnostics ::::::::::::::::::::::: 76 3.2.4 Blending ::::::::::::::::::::::::::::: 76 3.2.5 Modeling on the Counts Scale ::::::::::::::::: 77 3.3 Outbreak Detection ::::::::::::::::::::::::::: 78 3.3.1 Detection Method :::::::::::::::::::::::: 78 3.3.2 Outbreak Model :::::::::::::::::::::::: 79 3.3.3 Comparison of Methods :::::::::::::::::::: 80 3.3.4 Outbreak Injection Scenarios :::::::::::::::::: 82 3.3.5 Choosing Cutoff Limits ::::::::::::::::::::: 83 3.4 Results :::::::::::::::::::::::::::::::::: 84 3.4.1 Results Summary :::::::::::::::::::::::: 84 3.4.2 Examining the Difference in Detection Performance ::::: 86 3.4.3 False Positives :::::::::::::::::::::::::: 87 3.5 Discussion :::::::::::::::::::::::::::::::: 87 3.6 Conclusions ::::::::::::::::::::::::::::::: 88 3.7 Other Work ::::::::::::::::::::::::::::::: 89 4 A New Density Estimation Method: Ed ::::::::::::::::::: 91 4.1 Introduction ::::::::::::::::::::::::::::::: 91 vi Page 4.1.1 Background ::::::::::::::::::::::::::: 91 4.1.2 Summary of Ed ::::::::::::::::::::::::: 92 4.1.3 Overview of the Chapter :::::::::::::::::::: 93 4.2 Examples :::::::::::::::::::::::::::::::: 94 4.3 Estimation Step 1: Raw Estimates :::::::::::::::::: 95 4.3.1 Choosing κ ::::::::::::::::::::::::::: 97 4.3.2 Approximate Distribution of the Raw Estimates ::::::: 98 4.4 Estimation Step 2: Loess Smoothing ::::::::::::::::: 100 4.5 Diagnostic Checking, Model Selection, and Inference ::::::::: 101 4.5.1 Evaluating Goodness-of-Fit :::::::::::::::::: 101 4.5.2 Mallows Cp Model Selection :::::::::::::::::: 102 4.5.3 Validating Model Assumptions ::::::::::::::::: 102 4.5.4 Inference ::::::::::::::::::::::::::::: 102 4.6 Comparison with Kernel Density Estimates :::::::::::::: 103 4.6.1 The Family-Income data :::::::::::::::::::: 104 4.6.2 The Packet-Delay Data ::::::::::::::::::::: 105 4.6.3 The Normal-Mixture Data ::::::::::::::::::: 105 4.7 Grid Augmentation ::::::::::::::::::::::::::: 106 4.8 More Examples ::::::::::::::::::::::::::::: 108 4.9 Computational Methods :::::::::::::::::::::::: 109 4.10 Discussion :::::::::::::::::::::::::::::::: 110 5 Figures ::::::::::::::::::::::::::::::::::::: 113 5.1 Figures for Local Regression Modeling :::::::::::::::: 113 5.2 Figures for Time Series Modeling ::::::::::::::::::: 127 5.3 Figures for STL Advancements :::::::::::::::::::: 139 5.4 Figures for Syndromic Surveillance :::::::::::::::::: 179 5.5 Figures for Ed :::::::::::::::::::::::::::::: 205 Appendix: R Package Documentation ::::::::::::::::::::: 243 LIST OF REFERENCES :::::::::::::::::::::::::::: 273 VITA ::::::::::::::::::::::::::::::::::::::: 279 vii LIST OF TABLES Table Page (λ) 2.1 Coefficient ηi;j values for λ, i; j = 0; 1; 2 :::::::::::::::::: 47 3.1 Overall percentage of outbreaks detected by method ::::::::::: 85 viii LIST OF FIGURES Figure Page 5.1 Data for loess example :::::::::::::::::::::::::: 113 5.2 Loess example - fit at x = 0:5 :::::::::::::::::::::: 114 5.3 Loess fit for λ = 2, α = 0:4 :::::::::::::::::::::::: 115 5.4 Variance coefficient along the design space for the density loess fit :: 116 5.5 Loess fitted values for several parameters :::::::::::::::: 117 5.6 Loess residuals for several parameters :::::::::::::::::: 118 5.7 Mallows' Cp plot for the density data :::::::::::::::::: 119 5.8 Residual variance vs. ν for density data fits :::::::::::::: 120 5.9 Smoothing parameter α vs. equivalent number of parameters ν for den- sity data :::::::::::::::::::::::::::::::::: 121 5.10 Normal quantile plot for the density data loess fit with λ = 2, α = 0:4 122 5.11 Residual serial correlation plot for density data loess fit :::::::: 123 5.12 Residuals vs. design points for density data loess fit :::::::::: 124 5.13 Residuals vs. fitted values for density data loess fit :::::::::: 125 5.14 Loess fit for density data with exact fits at k-d tree vertices and inter- polation elsewhere ::::::::::::::::::::::::::::: 126 5.15 Two realizations of a unit-variance random walk process ::::::: 127 5.16 Time series loess fit to simulated data :::::::::::::::::: 128 5.17 Power transfer functions for symmetric loess operators :::::::: 129 5.18 CO2 concentration measurements at Mauna Loa, Hawaii from 1959 to 1997 :::::::::::::::::::::::::::::::::::: 130 5.19 STL decomposition for CO2 measurement time series ::::::::: 131 5.20 Power transfer function for each smoothing component of the CO2 STL decomposition ::::::::::::::::::::::::::::::: 132 5.21 Seasonal diagnostic plot for CO2 decomposition :::::::::::: 133 ix Figure Page 5.22 Remainder component by cycle-subseries for CO2 decomposition ::: 134 5.23 Fitted seasonal component by cycle-subseries for CO2 data :::::: 135 5.24 Trend diagnostic plot for CO2 decomposition :::::::::::::: 136 5.25 CO2 decomposition with low-frequency post-trend smoothing ::::: 137 5.26 CO2 decomposition with two post-trend smoothing components ::: 138 5.27 Variance coefficients for CO2 decomposition components ::::::: 139 5.28 Power transfer functions for STL operator matrix L∗ ::::::::: 140 5.29 Autocorrelation function for CO2 remainder term and residual term after ARMA modeling ::::::::::::::::::::::::::::: 141 5.30 Normal quantile plot of remainder term of CO2 model after ARMA mod- eling on remainder term ::::::::::::::::::::::::: 142 5.31 Predicting the CO2 series ahead 3 years with 95% prediction intervals 143 5.32 Simulated data to mimic daily visit counts to an emergency department (ED) :::::::::::::::::::::::::::::::::::: 144 5.33 Cp plot for CO2 decomposition with varying seasonal span n(s). ::: 145 5.34 Critical frequencies of a symmetric loess smooth vs. q−1 :::::::: 146 5.35 Residuals from fitting critical frequencies vs. q−1 for a linear and quadratic model ::::::::::::::::::::::::::::::::::: 147 5.36 Coefficients from quadratic polynomial fits of critical frequency vs. q−1 plotted against ! by λ :::::::::::::::::::::::::: 148 (lower) (upper) 5.37 Plot of n(p)fh and n(p)fh vs. log2 n(p) for λ(l) = 0; 1; 2 and ! = 0:05 :::::::::::::::::::::::::::::::::: 149 5.38 Variance coefficients
Load more

Recommended publications
  • Summary and Analysis of Extension Program Evaluation in R
    SUMMARY AND ANALYSIS OF EXTENSION PROGRAM EVALUATION IN R SALVATORE S. MANGIAFICO Rutgers Cooperative Extension New Brunswick, NJ VERSION 1.18.8 i ©2016 by Salvatore S. Mangiafico. Non-commercial reproduction of this content, with attribution, is permitted. For-profit reproduction without permission is prohibited. If you use the code or information in this site in a published work, please cite it as a source. Also, if you are an instructor and use this book in your course, please let me know. [email protected] Mangiafico, S.S. 2016. Summary and Analysis of Extension Program Evaluation in R, version 1.18.8. rcompanion.org/documents/RHandbookProgramEvaluation.pdf . Web version: rcompanion.org/handbook/ ii List of Chapters List of Chapters List of Chapters .......................................................................................................................... iii Table of Contents ...................................................................................................................... vi Introduction ............................................................................................................................... 1 Purpose of This Book.......................................................................................................................... 1 Author of this Book and Version Notes ............................................................................................... 2 Using R .............................................................................................................................................
    [Show full text]
  • Chapter 5 Local Regression Trees
    Chapter 5 Local Regression Trees In this chapter we explore the hypothesis of improving the accuracy of regression trees by using smoother models at the tree leaves. Our proposal consists of using local regression models to improve this smoothness. We call the resulting hybrid models, local regression trees. Local regression is a non-parametric statistical methodology that provides smooth modelling by not assuming any particular global form of the unknown regression function. On the contrary these models fit a functional form within the neighbourhood of the query points. These models are known to provide highly accurate predictions over a wide range of problems due to the absence of a “pre-defined” functional form. However, local regression techniques are also known by their computational cost, low comprehensibility and storage requirements. By integrating these techniques with regression trees, not only we improve the accuracy of the trees, but also increase the computational efficiency and comprehensibility of local models. In this chapter we describe the use of several alternative models for the leaves of regression trees. We study their behaviour in several domains an present their advantages and disadvantages. We show that local regression trees improve significantly the accuracy of “standard” trees at the cost of some additional computational requirements and some loss of comprehensibility. Moreover, local regression trees are also more accurate than trees that use linear models in the leaves. We have also observed that 167 168 CHAPTER 5. LOCAL REGRESSION TREES local regression trees are able to improve the accuracy of local modelling techniques in some domains. Compared to these latter models our local regression trees improve the level of computational efficiency allowing the application of local modelling techniques to large data sets.
    [Show full text]
  • A Tutorial on Bayesian Multi-Model Linear Regression with BAS and JASP
    Behavior Research Methods https://doi.org/10.3758/s13428-021-01552-2 A tutorial on Bayesian multi-model linear regression with BAS and JASP Don van den Bergh1 · Merlise A. Clyde2 · Akash R. Komarlu Narendra Gupta1 · Tim de Jong1 · Quentin F. Gronau1 · Maarten Marsman1 · Alexander Ly1,3 · Eric-Jan Wagenmakers1 Accepted: 21 January 2021 © The Author(s) 2021 Abstract Linear regression analyses commonly involve two consecutive stages of statistical inquiry. In the first stage, a single ‘best’ model is defined by a specific selection of relevant predictors; in the second stage, the regression coefficients of the winning model are used for prediction and for inference concerning the importance of the predictors. However, such second-stage inference ignores the model uncertainty from the first stage, resulting in overconfident parameter estimates that generalize poorly. These drawbacks can be overcome by model averaging, a technique that retains all models for inference, weighting each model’s contribution by its posterior probability. Although conceptually straightforward, model averaging is rarely used in applied research, possibly due to the lack of easily accessible software. To bridge the gap between theory and practice, we provide a tutorial on linear regression using Bayesian model averaging in JASP, based on the BAS package in R. Firstly, we provide theoretical background on linear regression, Bayesian inference, and Bayesian model averaging. Secondly, we demonstrate the method on an example data set from the World Happiness Report. Lastly,
    [Show full text]
  • Comparation on Several Smoothing Methods in Nonparametric Regression
    Comparation on Several Smoothing Methods in Nonparametric Regression R. Rizal Isnanto Abstract – There are three nonparametric regression Smoothing Method, completed with its variants will be methods covered in this section. These are Moving also analyzed. The advantages of all above methods are Average Filtering-Based Smoothing, Local Regression discussed as well as the disadvantages of the methods. Smoothing, and Kernel Smoothing Methods. The Moving Average Filtering-Based Smoothing methods discussed here are Moving Average Filtering and Savitzky-Golay Filtering. While, the Local Regression Smoothing II. MOVING AVERAGE FILTERING -BASED SMOOTHING techniques involved here are Lowess and Loess. In this type of smoothing, Robust Smoothing and Upper-and- A moving average is defined as an artificially Lower Smoothing are also explained deeply, related to constructed time series in which the value for a given Lowess and Loess. Finally, the Kernel Smoothing Method time period is replaced by the mean of that value and involves three methods discussed. These are Nadaraya- the values for some number of preceding and Watson Estimator, Priestley-Chao Estimator, and Local succeeding time periods [6]. Linear Kernel Estimator. The advantages of all above Moving average filtering is the former of smoothing methods are discussed as well as the disadvantages of the techniques. A moving average filter smooths data by methods. replacing each data point with the average of the Keywords : nonparametric regression , smoothing, moving neighboring data points defined within the span. This average, estimator, curve construction. process is equivalent to lowpass filtering with the response of the smoothing given by the difference I. INTRODUCTION equation: Nowadays, maybe the names “lowess” and “loess” which are derived from the term “locally weighted scatterplot smoothing,” as both methods use locally where ys(i) is the smoothed value for the ith data point, weighted linear regression to smooth data, is often N is the number of neighboring data points on either discussed.
    [Show full text]
  • Arxiv:1807.03931V2 [Cs.LG] 1 Aug 2018 E-Mail: [email protected] A
    Noname manuscript No. (will be inserted by the editor) A Hierarchical Bayesian Linear Regression Model with Local Features for Stochastic Dynamics Approximation Behnoosh Parsa · Keshav Rajasekaran · Franziska Meier · Ashis G. Banerjee Received: date / Accepted: date Abstract One of the challenges with model-based control of stochastic dynamical systems is that the state transition dynamics are involved, making it difficult and inefficient to make good-quality predictions of the states. Moreover, there are not many representational models for the majority of autonomous systems, as it is not easy to build a compact model that captures all the subtleties and uncertainties in the system dynamics. In this work, we present a hierarchical Bayesian linear regres- sion model with local features to learn the dynamics of such systems. The model is hierarchical since we consider non-stationary priors for the model parameters which increases its flexibility. To solve the maximum likelihood (ML) estimation problem for this hierarchical model, we use the variational expectation maximiza- tion (EM) algorithm, and enhance the procedure by introducing hidden target variables. The algorithm is guaranteed to converge to the optimal log-likelihood values under certain reasonable assumptions. It also yields parsimonious model structures, and consistently provides fast and accurate predictions for all our ex- amples, including two illustrative systems and a challenging micro-robotic system, involving large training and test sets. These results demonstrate the effectiveness of the method in approximating stochastic dynamics, which make it suitable for future use in a paradigm, such as model-based reinforcement learning, to compute optimal control policies in real time. B.
    [Show full text]
  • Incremental Local Gaussian Regression
    Incremental Local Gaussian Regression Franziska Meier1 Philipp Hennig2 Stefan Schaal1;2 [email protected] [email protected] [email protected] 1University of Southern California 2Max Planck Institute for Intelligent Systems Los Angeles, CA 90089, USA Spemannstraße 38, Tubingen,¨ Germany Abstract Locally weighted regression (LWR) was created as a nonparametric method that can approximate a wide range of functions, is computationally efficient, and can learn continually from very large amounts of incrementally collected data. As an interesting feature, LWR can regress on non-stationary functions, a beneficial property, for instance, in control problems. However, it does not provide a proper generative model for function values, and existing algorithms have a variety of manual tuning parameters that strongly influence bias, variance and learning speed of the results. Gaussian (process) regression, on the other hand, does provide a generative model with rather black-box automatic parameter tuning, but it has higher computational cost, especially for big data sets and if a non-stationary model is required. In this paper, we suggest a path from Gaussian (process) regression to locally weighted regression, where we retain the best of both approaches. Using a localizing function basis and approximate inference techniques, we build a Gaussian (process) regression algorithm of increasingly local nature and similar computational complexity to LWR. Empirical evaluations are performed on several synthetic and real robot datasets of increasing complexity and (big) data scale, and demonstrate that we consistently achieve on par or superior performance compared to current state-of-the-art methods while retaining a principled approach to fast incremental regression with minimal manual tuning parameters.
    [Show full text]
  • Locally Weighted Polynomial Regression: Parameter Choice and Application to Forecasts of the Great Salt Lake
    Utah State University DigitalCommons@USU Reports Utah Water Research Laboratory January 1995 Locally Weighted Polynomial Regression: Parameter Choice and Application to Forecasts of the Great Salt Lake Upmanu Lall Young-II Moon Ken Bosworth Follow this and additional works at: https://digitalcommons.usu.edu/water_rep Part of the Civil and Environmental Engineering Commons, and the Water Resource Management Commons Recommended Citation Lall, Upmanu; Moon, Young-II; and Bosworth, Ken, "Locally Weighted Polynomial Regression: Parameter Choice and Application to Forecasts of the Great Salt Lake" (1995). Reports. Paper 243. https://digitalcommons.usu.edu/water_rep/243 This Report is brought to you for free and open access by the Utah Water Research Laboratory at DigitalCommons@USU. It has been accepted for inclusion in Reports by an authorized administrator of DigitalCommons@USU. For more information, please contact [email protected]. LOCALLY WEIGHTED POLYNOMIAL REGRESSION: PARAMETER CHOICE AND APPLICATION TO FORECASTS OF THE GREAT SALT LAKE By: Upmanu Lall, Young-II Moon and Ken Bosworth Working Paper WP-95-HWR-UL/014 May 1995 Utah Water Research Laboratory Utah State University Logan, UT 84322-8200 (801) 797-3155 Fax (801) 797-3663 Locally Weighted Polynomial Regression: Parameter Choice and Application to Forecasts of the Great Salt Lake by Upmanu Lall and Young-n Moon Utah Water Research Lab. and Dept. of Civil and Environ. Eng. Utah State University, Logan UT 84322-8200 and Ken Bosworth Department of Mathematics, Idaho State University, Pocatello ID 83209-8085 Abstract Relationships between hydrologic variables are often nonlinear. Usually the functional form of such a relationship is not known a priori.
    [Show full text]
  • Baseline Subtraction Using Robust Local Regression Estimation
    Journal of Quantitative Spectroscopy & Radiative Transfer 68 (2001) 179}193 Baseline subtraction using robust local regression estimation Andreas F. Ruckstuhl! *, Matthew P. Jacobson", Robert W. Field", James A. Dodd# !Departement P, Zurcher Hochschule Winterthur (Zurich University of Applied Sciences), Postfach 805, 8401 Winterthur, Switzerland "Department of Chemistry and G.R. Harrison Spectroscopy Laboratory, Massachusetts Institute of Technology, Cambridge, MA 02139, USA #Space Vehicles Directorate, Air Force Research Laboratory, Hanscom Air Force Base, MA 01731-3010, USA Received 6 September 1999 Abstract A technique entitled robust baseline estimation is introduced, which uses techniques of robust local regression to estimate baselines in spectra that consist of sharp features superimposed upon a continuous, slowly varying baseline. The technique is applied to synthetic spectra, to evaluate its capabilities, and to laser-induced #uorescence spectra of OH (produced from the reaction of ozone with hydrogen atoms). The latter example is a particularly challenging case for baseline estimation because the experimental noise varies as a function of frequency. ( 2000 Elsevier Science Ltd. All rights reserved. Keywords: Baseline removal; LOWESS; Robust "tting; Smoothing 1. Introduction The robust baseline estimation (RBE) technique that is introduced in this paper is a technique for baseline removal; that is, in spectra that consist of sharp features superimposed upon a continuous, slowly varying baseline, it is designed to permit the separation of the two components, spectrum and baseline. There exist, of course, numerous techniques for baseline removal from spectra, because the problem is ubiquitous within spectroscopy. In many cases, baselines are simply removed `by eyea. There is nothing inherently wrong with such an approach, but we believe that, even for a `mundanea task such as baseline removal, it is bene"cial to use numerical tools that minimize the need for judgement calls and permit reproduction of the results by others.
    [Show full text]
  • Automating the Smoothing of Time Series Data Shilpy Sharma*, David a Swayne and Charlie Obimbo School of Computer Science, University of Guelph, Canada
    ntal & A me na n ly o t ir ic v a n l Sharma et al., J Environ Anal Toxicol 2015, 5:5 T E o Journal of f x o i l c o a DOI: 10.4172/2161-0525.1000304 n l o r g u y o J Environmental & Analytical Toxicology ISSN: 2161-0525 ResearchResearch Article Article OpenOpen Access Access Automating the Smoothing of Time Series Data Shilpy Sharma*, David A Swayne and Charlie Obimbo School of Computer Science, University of Guelph, Canada Abstract Modelling requires comparison of model outputs to measurements, for calibration and verification. A key aspect data smoothing is to “filter out” noise. Often, data must be adjusted to a model’s time step (e.g. hourly to daily). For noisy data, LOWESS/LOESS (Locally Weighted Scatterplot Smoothing) is a popular piecewise regression technique. It produces a “smoothed” time series. LOWESS/LOESS is often used to visually assess the relationship between variables. The selection of LOWESS tuning parameters is usually performed on a visual trial and error basis. We investigate the so-called robust AIC (Akaike Information Criteria) for automatic selection of smoothing. Robust Pearson correlation coefficient and mean-squared error are employed to determine the polynomial degree of piecewise regression. The exclusion of outliers is attempted using a Hampel outlier identifier. We illustrate, assuming noisy linear data, how our proposed methods work for auto-tuning both the smoothing parameter and the degree of polynomial for LOWESS/ LOESS. Keywords: LOWESS; LOESS; Data smoothing most efficient and robust method for identification of outliers [9].
    [Show full text]
  • Econstor Wirtschaft Leibniz Information Centre Make Your Publications Visible
    A Service of Leibniz-Informationszentrum econstor Wirtschaft Leibniz Information Centre Make Your Publications Visible. zbw for Economics Loader, Catherine Working Paper Smoothing: Local Regression Techniques Papers, No. 2004,12 Provided in Cooperation with: CASE - Center for Applied Statistics and Economics, Humboldt University Berlin Suggested Citation: Loader, Catherine (2004) : Smoothing: Local Regression Techniques, Papers, No. 2004,12, Humboldt-Universität zu Berlin, Center for Applied Statistics and Economics (CASE), Berlin This Version is available at: http://hdl.handle.net/10419/22186 Standard-Nutzungsbedingungen: Terms of use: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Documents in EconStor may be saved and copied for your Zwecken und zum Privatgebrauch gespeichert und kopiert werden. personal and scholarly purposes. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle You are not to copy documents for public or commercial Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich purposes, to exhibit the documents publicly, to make them machen, vertreiben oder anderweitig nutzen. publicly available on the internet, or to distribute or otherwise use the documents in public. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, If the documents have been made available under an Open gelten abweichend von diesen Nutzungsbedingungen die in der dort Content Licence (especially Creative Commons Licences), you genannten Lizenz gewährten Nutzungsrechte. may exercise further usage rights as specified in the indicated licence. www.econstor.eu Smoothing: Local Regression Techniques Catherine Loader1 Department of Statistics, Case Western Reserve University, Cleveland, OH 44106, USA. [email protected] Smoothing methods attempt to find functional relationships between different measurements. As in the standard regression setting, the data is assumed to consist of measurements of a response variable, and one or more predictor variables.
    [Show full text]
  • The LOESS Procedure
    Chapter 38 The LOESS Procedure Chapter Table of Contents OVERVIEW ...................................1855 Local Regression and the Loess Method ....................1855 GETTING STARTED ..............................1856 ScatterPlotSmoothing.............................1856 SYNTAX .....................................1865 PROCLOESSStatement............................1866 BYStatement..................................1866 IDStatement..................................1867 MODELStatement...............................1867 SCORE Statement ...............................1870 WEIGHTStatement..............................1871 DETAILS .....................................1871 MissingValues.................................1871 OutputDataSets................................1871 DataScaling...................................1873 Direct versus Interpolated Fitting . ......................1874 kdTreesandBlending.............................1874 LocalWeighting.................................1875 IterativeReweighting..............................1875 Specifying the Local Polynomials . ......................1875 StatisticalInference...............................1876 ScoringDataSets................................1876 ODSTableNames...............................1877 EXAMPLES ...................................1877 Example38.1EngineExhaustEmissions...................1877 Example 38.2 Sulfate Deposits in the USA for 1990 . ...........1885 Example38.3CatalystExperiment.......................1891 Example38.4AutomaticSmoothingParameterSelection...........1893
    [Show full text]
  • Extending Linear Regression: Weighted Least Squares and Local Regression
    Extending Linear Regression: Weighted Least Squares and Local Regression 36-350, Data Mining 10 October 2008 1 Weighted Least Squares Instead of minimizing the residual sum of squares, n X 2 (yi − ~xi · β) (1) i=1 we could minimize the weighted sum of squares, n X 2 wi(yi − ~xi · β) (2) i=1 This includes ordinary least squares as the special case where all the weights wi = 1. (Exercise: Is that a necessary as well as a sufficient condition for (2) and (1) to have the same minimum?) We can solve it by the same kind of algebra we used to solve the ordinary linear least squares problem. But why would we want to solve it? For three reasons. 1. Focusing accuracy. We may care very strongly about predicting the re- sponse for certain values of the input — ones we expect to see often again, ones where mistakes are especially costly or embarrassing or painful, etc. — than others. If we give the points xi near that region big weights wi, and points elsewhere smaller weights, the regression will be pulled towards matching the data in that region. 2. Discounting imprecision. Ordinary least squares is the maximum likeli- hood estimate when the in Y = X~ · β + is IID Gaussian white noise. This means that the variance of has to be constant, and we measure the regression curve with the same precision elsewhere. This situation, of constant noise variance, is called homoskedasticity. Often however the magnitude of the noise is not constant, and the data are heteroskedastic. When we have heteroskedasticity, even if each noise term is still Gaussian, ordinary least squares is no longer the maximum likelihood estimate, and 1 2 so no longer efficient.
    [Show full text]