Nonlinear Regression Functions
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Automatic Linearity Detection
View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Mathematical Institute Eprints Archive Report no. [13/04] Automatic linearity detection Asgeir Birkisson a Tobin A. Driscoll b aMathematical Institute, University of Oxford, 24-29 St Giles, Oxford, OX1 3LB, UK. [email protected]. Supported for this work by UK EPSRC Grant EP/E045847 and a Sloane Robinson Foundation Graduate Award associated with Lincoln College, Oxford. bDepartment of Mathematical Sciences, University of Delaware, Newark, DE 19716, USA. [email protected]. Supported for this work by UK EPSRC Grant EP/E045847. Given a function, or more generally an operator, the question \Is it lin- ear?" seems simple to answer. In many applications of scientific computing it might be worth determining the answer to this question in an automated way; some functionality, such as operator exponentiation, is only defined for linear operators, and in other problems, time saving is available if it is known that the problem being solved is linear. Linearity detection is closely connected to sparsity detection of Hessians, so for large-scale applications, memory savings can be made if linearity information is known. However, implementing such an automated detection is not as straightforward as one might expect. This paper describes how automatic linearity detection can be implemented in combination with automatic differentiation, both for stan- dard scientific computing software, and within the Chebfun software system. The key ingredients for the method are the observation that linear operators have constant derivatives, and the propagation of two logical vectors, ` and c, as computations are carried out. -
Transformations, Polynomial Fitting, and Interaction Terms
FEEG6017 lecture: Transformations, polynomial fitting, and interaction terms [email protected] The linearity assumption • Regression models are both powerful and useful. • But they assume that a predictor variable and an outcome variable are related linearly. • This assumption can be wrong in a variety of ways. The linearity assumption • Some real data showing a non- linear connection between life expectancy and doctors per million population. The linearity assumption • The Y and X variables here are clearly related, but the correlation coefficient is close to zero. • Linear regression would miss the relationship. The independence assumption • Simple regression models also assume that if a predictor variable affects the outcome variable, it does so in a way that is independent of all the other predictor variables. • The assumed linear relationship between Y and X1 is supposed to hold no matter what the value of X2 may be. The independence assumption • Suppose we're trying to predict happiness. • For men, happiness increases with years of marriage. For women, happiness decreases with years of marriage. • The relationship between happiness and time may be linear, but it would not be independent of sex. Can regression models deal with these problems? • Fortunately they can. • We deal with non-linearity by transforming the predictor variables, or by fitting a polynomial relationship instead of a straight line. • We deal with non-independence of predictors by including interaction terms in our models. Dealing with non-linearity • Transformation is the simplest method for dealing with this problem. • We work not with the raw values of X, but with some arbitrary function that transforms the X values such that the relationship between Y and f(X) is now (closer to) linear. -
Glossary: a Dictionary for Linear Algebra
GLOSSARY: A DICTIONARY FOR LINEAR ALGEBRA Adjacency matrix of a graph. Square matrix with aij = 1 when there is an edge from node T i to node j; otherwise aij = 0. A = A for an undirected graph. Affine transformation T (v) = Av + v 0 = linear transformation plus shift. Associative Law (AB)C = A(BC). Parentheses can be removed to leave ABC. Augmented matrix [ A b ]. Ax = b is solvable when b is in the column space of A; then [ A b ] has the same rank as A. Elimination on [ A b ] keeps equations correct. Back substitution. Upper triangular systems are solved in reverse order xn to x1. Basis for V . Independent vectors v 1,..., v d whose linear combinations give every v in V . A vector space has many bases! Big formula for n by n determinants. Det(A) is a sum of n! terms, one term for each permutation P of the columns. That term is the product a1α ··· anω down the diagonal of the reordered matrix, times det(P ) = ±1. Block matrix. A matrix can be partitioned into matrix blocks, by cuts between rows and/or between columns. Block multiplication of AB is allowed if the block shapes permit (the columns of A and rows of B must be in matching blocks). Cayley-Hamilton Theorem. p(λ) = det(A − λI) has p(A) = zero matrix. P Change of basis matrix M. The old basis vectors v j are combinations mijw i of the new basis vectors. The coordinates of c1v 1 +···+cnv n = d1w 1 +···+dnw n are related by d = Mc. -
A Toolbox for Nonlinear Regression in R: the Package Nlstools
JSS Journal of Statistical Software August 2015, Volume 66, Issue 5. http://www.jstatsoft.org/ A Toolbox for Nonlinear Regression in R: The Package nlstools Florent Baty Christian Ritz Sandrine Charles Cantonal Hospital St. Gallen University of Copenhagen University of Lyon Martin Brutsche Jean-Pierre Flandrois Cantonal Hospital St. Gallen University of Lyon Marie-Laure Delignette-Muller University of Lyon Abstract Nonlinear regression models are applied in a broad variety of scientific fields. Various R functions are already dedicated to fitting such models, among which the function nls() has a prominent position. Unlike linear regression fitting of nonlinear models relies on non-trivial assumptions and therefore users are required to carefully ensure and validate the entire modeling. Parameter estimation is carried out using some variant of the least- squares criterion involving an iterative process that ideally leads to the determination of the optimal parameter estimates. Therefore, users need to have a clear understanding of the model and its parameterization in the context of the application and data consid- ered, an a priori idea about plausible values for parameter estimates, knowledge of model diagnostics procedures available for checking crucial assumptions, and, finally, an under- standing of the limitations in the validity of the underlying hypotheses of the fitted model and its implication for the precision of parameter estimates. Current nonlinear regression modules lack dedicated diagnostic functionality. So there is a need to provide users with an extended toolbox of functions enabling a careful evaluation of nonlinear regression fits. To this end, we introduce a unified diagnostic framework with the R package nlstools. -
Chapter 1: Linearity: Basic Concepts and Examples
CHAPTER I LINEARITY: BASIC CONCEPTS AND EXAMPLES In this chapter we start with the concept of general linear spaces with elements in it called vectors, for “setting up the stage”. Then we introduce “actors” called linear mappings, which act upon vectors. In the mathematical literature, “vector spaces” is synonymous to “linear spaces” and these words will be used exchangeably. Also, “linear transformations” and “linear mappings” or simply “linear maps”, are also synonymous. 1. Linear Spaces and Linear Maps § 1.1. A vector space is an entity containing objects called vectors. A vector is usually conceived to be something which has a magnitude and direction, so that it can be drawn as an arrow: You can add or subtract two vectors: You can also multiply vectors by scalars: Such things should be familiar to you. However, we should not be so narrow-minded to think that only those objects repre- 1 sented geometrically by arrows in a 2D or 3D space can be regarded as vectors. As long as we have a collection of objects among which two algebraic operations called addition and scalar multiplication can be performed, so that certain rules of such operations are obeyed, we may regard this collection as a vector space and call the objects in this col- lection vectors. Our definition of vector spaces should be so general that we encounter vector spaces almost everyday and almost everywhere. Examples of vector spaces include many spaces of functions, spaces of polynomials, spaces of sequences etc. (in addition to the well-known 3D space in which vectors are represented as arrows.) The universality and the omnipresence of vector spaces is one good reason for placing linear algebra in the position of paramount importance in basic mathematics. -
An Efficient Nonlinear Regression Approach for Genome-Wide
An Efficient Nonlinear Regression Approach for Genome-wide Detection of Marginal and Interacting Genetic Variations Seunghak Lee1, Aur´elieLozano2, Prabhanjan Kambadur3, and Eric P. Xing1;? 1School of Computer Science, Carnegie Mellon University, USA 2IBM T. J. Watson Research Center, USA 3Bloomberg L.P., USA [email protected] Abstract. Genome-wide association studies have revealed individual genetic variants associated with phenotypic traits such as disease risk and gene expressions. However, detecting pairwise in- teraction effects of genetic variants on traits still remains a challenge due to a large number of combinations of variants (∼ 1011 SNP pairs in the human genome), and relatively small sample sizes (typically < 104). Despite recent breakthroughs in detecting interaction effects, there are still several open problems, including: (1) how to quickly process a large number of SNP pairs, (2) how to distinguish between true signals and SNPs/SNP pairs merely correlated with true sig- nals, (3) how to detect non-linear associations between SNP pairs and traits given small sam- ple sizes, and (4) how to control false positives? In this paper, we present a unified framework, called SPHINX, which addresses the aforementioned challenges. We first propose a piecewise linear model for interaction detection because it is simple enough to estimate model parameters given small sample sizes but complex enough to capture non-linear interaction effects. Then, based on the piecewise linear model, we introduce randomized group lasso under stability selection, and a screening algorithm to address the statistical and computational challenges mentioned above. In our experiments, we first demonstrate that SPHINX achieves better power than existing methods for interaction detection under false positive control. -
Linearity in Calibration: How to Test for Non-Linearity Previous Methods for Linearity Testing Discussed in This Series Contain Certain Shortcom- Ings
Chemometrics in Spectroscopy Linearity in Calibration: How to Test for Non-linearity Previous methods for linearity testing discussed in this series contain certain shortcom- ings. In this installment, the authors describe a method they believe is superior to others. Howard Mark and Jerome Workman Jr. n the previous installment of “Chemometrics in Spec- instrumental methods have to produce a number, represent- troscopy” (1), we promised we would present a descrip- ing the final answer for that instrument’s quantitative assess- I tion of what we believe is the best way to test for linearity ment of the concentration, and that is the test result from (or non-linearity, depending upon your point of view). In the that instrument. This is a univariate concept to be sure, but first three installments of this column series (1–3) we exam- the same concept that applies to all other analytical methods. ined the Durbin–Watson (DW) statistic along with other Things may change in the future, but this is currently the way methods of testing for non-linearity. We found that while the analytical results are reported and evaluated. So the question Durbin–Watson statistic is a step in the right direction, but we to be answered is, for any given method of analysis: Is the also saw that it had shortcomings, including the fact that it relationship between the instrument readings (test results) could be fooled by data that had the right (or wrong!) charac- and the actual concentration linear? teristics. The method we present here is mathematically This method of determining non-linearity can be viewed sound, more subject to statistical validity testing, based upon from a number of different perspectives, and can be consid- well-known mathematical principles, consists of much higher ered as coming from several sources. -
Model Selection, Transformations and Variance Estimation in Nonlinear Regression
Model Selection, Transformations and Variance Estimation in Nonlinear Regression Olaf Bunke1, Bernd Droge1 and J¨org Polzehl2 1 Institut f¨ur Mathematik, Humboldt-Universit¨at zu Berlin PSF 1297, D-10099 Berlin, Germany 2 Konrad-Zuse-Zentrum f¨ur Informationstechnik Heilbronner Str. 10, D-10711 Berlin, Germany Abstract The results of analyzing experimental data using a parametric model may heavily depend on the chosen model. In this paper we propose procedures for the ade- quate selection of nonlinear regression models if the intended use of the model is among the following: 1. prediction of future values of the response variable, 2. estimation of the un- known regression function, 3. calibration or 4. estimation of some parameter with a certain meaning in the corresponding field of application. Moreover, we propose procedures for variance modelling and for selecting an appropriate nonlinear trans- formation of the observations which may lead to an improved accuracy. We show how to assess the accuracy of the parameter estimators by a ”moment oriented bootstrap procedure”. This procedure may also be used for the construction of confidence, prediction and calibration intervals. Programs written in Splus which realize our strategy for nonlinear regression modelling and parameter estimation are described as well. The performance of the selected model is discussed, and the behaviour of the procedures is illustrated by examples. Key words: Nonlinear regression, model selection, bootstrap, cross-validation, variable transformation, variance modelling, calibration, mean squared error for prediction, computing in nonlinear regression. AMS 1991 subject classifications: 62J99, 62J02, 62P10. 1 1 Selection of regression models 1.1 Preliminary discussion In many papers and books it is discussed how to analyse experimental data estimating the parameters in a linear or nonlinear regression model, see e.g. -
Nonlinear Regression, Nonlinear Least Squares, and Nonlinear Mixed Models in R
Nonlinear Regression, Nonlinear Least Squares, and Nonlinear Mixed Models in R An Appendix to An R Companion to Applied Regression, third edition John Fox & Sanford Weisberg last revision: 2018-06-02 Abstract The nonlinear regression model generalizes the linear regression model by allowing for mean functions like E(yjx) = θ1= f1 + exp[−(θ2 + θ3x)]g, in which the parameters, the θs in this model, enter the mean function nonlinearly. If we assume additive errors, then the parameters in models like this one are often estimated via least squares. In this appendix to Fox and Weisberg (2019) we describe how the nls() function in R can be used to obtain estimates, and briefly discuss some of the major issues with nonlinear least squares estimation. We also describe how to use the nlme() function in the nlme package to fit nonlinear mixed-effects models. Functions in the car package than can be helpful with nonlinear regression are also illustrated. The nonlinear regression model is a generalization of the linear regression model in which the conditional mean of the response variable is not a linear function of the parameters. As a simple example, the data frame USPop in the carData package, which we load along with the car package, has decennial U. S. Census population for the United States (in millions), from 1790 through 2000. The data are shown in Figure 1 (a):1 library("car") Loading required package: carData brief(USPop) 22 x 2 data.frame (17 rows omitted) year population [i] [n] 1 1790 3.9292 2 1800 5.3085 3 1810 7.2399 .. -
Student Difficulties with Linearity and Linear Functions and Teachers
Student Difficulties with Linearity and Linear Functions and Teachers' Understanding of Student Difficulties by Valentina Postelnicu A Dissertation Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy Approved April 2011 by the Graduate Supervisory Committee: Carole Greenes, Chair Victor Pambuccian Finbarr Sloane ARIZONA STATE UNIVERSITY May 2011 ABSTRACT The focus of the study was to identify secondary school students' difficulties with aspects of linearity and linear functions, and to assess their teachers' understanding of the nature of the difficulties experienced by their students. A cross-sectional study with 1561 Grades 8-10 students enrolled in mathematics courses from Pre-Algebra to Algebra II, and their 26 mathematics teachers was employed. All participants completed the Mini-Diagnostic Test (MDT) on aspects of linearity and linear functions, ranked the MDT problems by perceived difficulty, and commented on the nature of the difficulties. Interviews were conducted with 40 students and 20 teachers. A cluster analysis revealed the existence of two groups of students, Group 0 enrolled in courses below or at their grade level, and Group 1 enrolled in courses above their grade level. A factor analysis confirmed the importance of slope and the Cartesian connection for student understanding of linearity and linear functions. There was little variation in student performance on the MDT across grades. Student performance on the MDT increased with more advanced courses, mainly due to Group 1 student performance. The most difficult problems were those requiring identification of slope from the graph of a line. That difficulty persisted across grades, mathematics courses, and performance groups (Group 0, and 1). -
LINEARITY Definition
SECTION 1.1 – LINEARITY Definition (Total Change) What that means: Algebraically Geometrically Example 1 At the beginning of the year, the price of gas was $3.19 per gallon. At the end of the year, the price of gas was $1.52 per gallon. What is the total change in the price of gas? Algebraically Geometrically 1 Definition (Average Rate of Change) What that means: Algebraically: Geometrically: Example 2 John collects marbles. After one year, he had 60 marbles and after 5 years, he had 140 marbles. What is the average rate of change of marbles with respect to time? Algebraically Geometrically 2 Definition (Linearly Related, Linear Model) Example 3 On an average summer day in a large city, the pollution index at 8:00 A.M. is 20 parts per million, and it increases linearly by 15 parts per million each hour until 3:00 P.M. Let P be the amount of pollutants in the air x hours after 8:00 A.M. a.) Write the linear model that expresses P in terms of x. b.) What is the air pollution index at 1:00 P.M.? c.) Graph the equation P for 0 ≤ x ≤ 7 . 3 SECTION 1.2 – GEOMETRIC & ALGEBRAIC PROPERTIES OF A LINE Definition (Slope) What that means: Algebraically Geometrically Definition (Graph of an Equation) 4 Example 1 Graph the line containing the point P with slope m. a.) P = ( ,1 1) & m = −3 c.) P = − ,1 −3 & m = 3 ( ) 5 b.) P = ,2 1 & m = 1 ( ) 2 d.) P = ,0 −2 & m = − 2 ( ) 3 5 Definition (Slope -Intercept Form) Definition (Point -Slope Form) 6 Example 2 Find the equation of the line through the points P and Q. -
The Gradient System. Understanding Gradients from an EM Perspective: (Gradient Linearity, Eddy Currents, Maxwell Terms & Peripheral Nerve Stimulation)
The Gradient System. Understanding Gradients from an EM Perspective: (Gradient Linearity, Eddy Currents, Maxwell Terms & Peripheral Nerve Stimulation) Franz Schmitt Siemens Healthcare, Business Unit MR Karl Schall Str. 6 91054 Erlangen, Germany Email: [email protected] 1 Introduction Since the introduction of magnetic resonance imaging (MRI) as a commercially available diagnostic tool in 1984, dramatic improvements have been achieved in all features defining image quality, such as resolution, signal to noise ratio (SNR), and speed. Initially, spin echo (SE)(1) images containing 128x128 pixels were measured in several minutes. Nowadays, standard matrix sizes for musculo skeletal and neuro studies using SE based techniques is 512x512 with similar imaging times. Additionally, the introduction of echo planar imaging (EPI)(2,3) techniques had made it possible to acquire 128x128 images as produced with earlier MRI scanners in roughly 100 ms. This im- provement in resolution and speed is only possible through improvements in gradient hardware of the recent MRI scanner generations. In 1984, typical values for the gradients were in the range of 1 - 2 mT/m at rise times of 1-2 ms. Over the last decade, amplitudes and rise times have changed in the order of magnitudes. Present gradient technology allows the application of gradient pulses up to 50 mT/m (for whole-body applications) with rise times down to 100 µs. Using dedicated gradient coils, such as head-insert gradient coils, even higher amplitudes and shorter rise times are feasible. In this presentation we demonstrate how modern high-performance gradient systems look like and how components of an MR system have to correlate with one another to guarantee maximum performance.