Generalized Least-Squares Regressions IV: Theory and Classification Using Generalized Means
Total Page:16
File Type:pdf, Size:1020Kb
City University of New York (CUNY) CUNY Academic Works Publications and Research Kingsborough Community College 2014 Generalized Least-Squares Regressions IV: Theory and Classification Using Generalized Means Nataniel Greene CUNY Kingsborough Community College How does access to this work benefit ou?y Let us know! More information about this work at: https://academicworks.cuny.edu/kb_pubs/77 Discover additional works at: https://academicworks.cuny.edu This work is made publicly available by the City University of New York (CUNY). Contact: [email protected] Mathematics and Computers in Science and Industry Generalized Least-Squares Regressions IV: Theory and Classication Using Generalized Means Nataniel Greene Department of Mathematics and Computer Science Kingsborough Community College, CUNY 2001 Oriental Boulevard, Brooklyn, NY 11235, USA Email: [email protected] Abstract— The theory of generalized least-squares is reformu- arithmetic mean of the square deviations in x and y and was lated here using the notion of generalized means. The generalized called Pythagorean regression previously. Here, logarithmic, least-squares problem seeks a line which minimizes the average Heronian, centroidal, identric, Lorentz, and root mean square generalized mean of the square deviations in x and y. The notion of a generalized mean is equivalent to the generating regressions are described for the rst time. Ordinary least- function concept of the previous papers but allows for a more squares regression is shown here to be equivalent to minimum robust understanding and has an already existing literature. or maximum mean regression. Regressions based on weighted Generalized means are applied to the task of constructing arithmetic means of order and weighted geometric means more examples, simplifying the theory, and further classifying generalized least-squares regressions. of order are explored. The weights and parameterize all generalized mean square regression lines lying between the Keywords—Linear regression, least-squares, orthogonal re- gression, geometric mean regression, generalized least-squares, two ordinary least-squares lines. generalized mean square regression. Power mean regression of order p offers a particularly simple framework for parameterizing all the generalized mean I. OVERVIEW square regressions previously described. The p-scale has Ordinary least-squares regression suffers from a fundamen- xed numerical values corresponding to many known special tal lack of symmetry: the regression line of y given x and the means. All the symmetric regressions discussed in the previous regression line of x given y are not inverses of each other. papers are power mean regressions for some value of p. Ordinary least-squares corresponds to p = . The power Ordinary least-squares y x regression minimizes the average 1 square deviation betweenj the data and the line in the y variable mean is one example of a generalized mean whose free and ordinary least-squares x y minimizes the average square parameter unites a variety of special means as subcases. Other deviation between the dataj and the line in the x variable. generalized means which do the same include: the Dietel- A theory of generalized least-squares was described by this Gordon mean of order r, Stolarsky's mean of order s, and author for minimizing the average of a symmetric function of Gini's mean of order t. There are also two-parameter means the square deviations in both x and y variables [7,8,9]. The due to Stolarsky and Gini. Regression formulas based on all symmetric function was referred to as a generating function these generalized means are worked out here for the rst time. for the particular regression method. II. REGRESSIONS BASED ON GENERALIZED MEANS This paper continues the development of the theory of generalized least-squares, reformulated using the notion of Generalized means are applied to the task of constructing generalized means. The generalized least-squares problem more examples, simplifying the theory, and further classifying described here seeks a line which minimizes the average generalized least-squares regressions. generalized mean of the square deviations in x and y. The A. Axioms of a Generalized Mean notion of a generalized mean is equivalent to the generating function concept of the previous papers but allows for a more The axioms of a generalized mean presented here are drawn robust understanding and has an already existing literature. from Mays [13] and also from Chen [2]. It is clear from the name that geometric mean regression Denition 1: A function M (x; y) denes a generalized (GMR) seeks a line which minimizes the average geometric mean for x; y > 0 if it satises Properties 1-5 below. If mean of the square deviations in x and y. Orthogonal it satises Property 6 it is called a homogenous generalized regression seeks a line which minimizes the average harmonic mean. The properties are: mean of the square deviations in x and y. Therefore it is also 1. (Continuity) M (x; y) is continuous in each variable. called harmonic mean regression (HMR). Arithmetic mean 2. (Monotonicity) M (x; y) is non-decreasing in each vari- regression (AMR) seeks a line which minimizes the average able. ISBN: 978-1-61804-247-7 19 Mathematics and Computers in Science and Industry 3. (Symmetry) M (x; y) = M (y; x) : or using Ehrenberg's formula 4. (Identity) M (x; x) = x: 2 5. (Intermediacy) min (x; y) M (x; y) max (x; y) : E = g (b) 2 1 2 + 2 b y y x 6. (Homogeneity) M (tx; ty) = tM (x; y) for all t > 0: x All known means are included in this denition. All 2 + a + bx y (5) the means discussed in this paper are homogeneous. The where reader can verify that the weighted arithmetic mean or convex 1 combination of any two generalized means is a generalized g (b) = M 1; : (6) b2 mean. The weighted geometric mean of any two generalized a 1 1 Proof: Substitute b +xi b yi with b (a + bxi yi) and means is a generalized mean. More generally, the generalized then use the homogeneity property: mean of any two generalized means is itself a generalized n 2 mean. 1 2 (a + bxi yi) E = M (a + bxi yi) ; n b2 The equivalence of generalized means and generating func- i=1 ! tions is now demonstrated. Xn 1 2 1 Theorem 1: Let M (x; y) be any generalized mean, then = (a + bxi yi) M 1; : n b2 2 2 i=1 (x; y) = M x ; y is the generating function for a X corresponding generalized symmetric least-squares regression. Dene 1 Let (x; y) be any generating function, then M (x; y) = g (b) = M 1; ; b2 x ; y denes a generalized mean. The weight j j j j 1 1 factor g (b) outside of the summation and replace using functionp isp given by g (b) = 1; b = M 1; b2 : From here it is clear that the theory of generalized least- Ehrenberg's formula. The theory now continues unchanged with all the same squares can be reformulated using generalized means. The theorems and formulas involving the weight function g (b). general symmetric least-squares problem is re-stated as fol- It is reviewed now. To nd the regression coefcients a and lows. Denition 2: (The General Symmetric Least-Squares Prob- b take rst order partial derivatives of the error function Ea and Eb and set them equal to zero. The result is a = b lem) Values of a and b are sought which minimize an error y x function dened by and the First Discrepancy Formula: 2 2 N 2 y 1 g0 (b) y y 2 1 2 a 1 b = b + 1 : E = M (a + bxi yi) ; + xi yi (1) 2 g (b) N b b x x x ! i=1 ! X (7) where M (x; y) is any generalized mean. The solution to this To derive the slope equation for any generalized regression problem is called generalized mean square regression. of interest, begin with the First Discrepancy Formula and Denition 3: (The General Weighted Ordinary Least- substitute the specic expression for g (b) into the formula, Squares Problem) Values of a and b are sought which minimize simplify and reset the equation equal to zero. What emerges an error function dened by is the specic slope equation. This is the procedure employed N for all the slope equations presented in this paper. 1 2 y E = g (b) (a + bxi yi) (2) Solving this equation for the discrepancy b using the N x i=1 quadratic formula yields the Second Discrepancy Formula: X or using Ehrenberg's formula g (b) 2 g (b) 2 2 b y = 1 1 2 y 0 : 2 2 2 y E = g (b) 1 + b x g0 (b) 0 s x g (b) 1 y x x (8) @ A 2 In order for the y-intercept a and slope b to minimize the error + a + bx y (3) function the Hessian determinant E E (E )2 must be aa bb ab where g (b) is a positive even function that is non-decreasing positive. Calculation of this determinant in the general case for b < 0 and non-increasing for b > 0. yields a function The next theorem states that every generalized mean square G (b) = 2g0(b)=g(b) g00(b)=g0(b) (9) regression problem is equivalent to a weighted ordinary least- squares problem with weight function g (b). called the indicative function. The Hessian determinant is pos- Theorem 2: The general symmetric least-squares error itive provided that G (b) b y > 1. This differential function can be written equivalently as x equation for the weight function is solved to yield N 1 2 E = g (b) (a + bxi yi) (4) g(b) = 1=(c + k exp( G(b)db)db): (10) N i=1 X Z Z ISBN: 978-1-61804-247-7 20 Mathematics and Computers in Science and Industry B.