City University of New York (CUNY) CUNY Academic Works

Publications and Research Kingsborough Community College

2016

Generalized Least-Powers Regressions I: Bivariate Regressions

Nataniel Greene CUNY Kingsborough Community College

How does access to this work benefit ou?y Let us know!

More information about this work at: https://academicworks.cuny.edu/kb_pubs/105 Discover additional works at: https://academicworks.cuny.edu

This work is made publicly available by the City University of New York (CUNY). Contact: [email protected] INTERNATIONAL JOURNAL OF MATHEMATICAL MODELS AND METHODS IN APPLIED SCIENCES Volume 10, 2016 Generalized Least-Powers Regressions I: Bivariate Regressions Nataniel Greene

of this paper is on even values of p, since this case allows Abstract— The bivariate theory of generalized least-squares is for the derivation of analytic formulas to nd the regression extended here to least-powers. The bivariate generalized least- coefcients, analogous to what was done for least-squares. powers problem of order p seeks a line which minimizes the average generalized mean of the absolute pth power deviations between the The numerical example presented illustrates that bivariate data and the line. Least-squares regressions utilize second order generalized least-powers methods perform comparably to gen- moments of the data to construct the regression line whereas least- eralized least-squares methods but have a greater range of powers regressions use moments of order p to construct the line. slope values. The focus is on even values of p, since this case admits analytic so- lution methods for the regression coefcients. A numerical example shows generalized least-powers methods performing comparably to II.BIVARIATE ORDINARY AND GENERALIZED generalized least-squares methods, but with a wider range of slope LEAST-POWERS REGRESSION values. A. Bivariate Ordinary Least-Powers Regression OLP Keywords—Least-powers, generalized least-powers, least-squares, p generalized least-squares, geometric mean regression, orthogonal The generalization of ordinary least-squares regression regression, least-quartic regression. (OLS) to arbitrary powers is called ordinary least-powers regression of order p and is denoted here by OLPp. OLS I.OVERVIEW is the same thing as OLP2. The case of OLP4, called least- quartic regression, is described in a paper by Arbia [1]. OR two variables x and y ordinary least-squares y x j Denition 1: (The Ordinary Least-Powers Problem) Values F regression suffers from a fundamental lack of symmetry. of a and b are sought which minimize an error function dened It minimizes the distance between the data and the regression by line in the dependent variable y alone. To predict the value N of the independent variable x one cannot simply solve for 1 p E = a + bxi yi : (1) N j j this variable using the regression equation. Instead one must i=1 derive a new regression equation treating x as the dependent X variable. This is called ordinary least-squares x y regression. The resulting line y = a + bx is called the ordinary least- j powers y x regression line. The fact that there are two ordinary least-squares lines to j model a single set of data is problematic. One wishes to The explicit bivariate formula for the ordinary least-squares have a single linear model for the data, for which it is valid error described by Ehrenberg [3] is generalized now to higher- to solve for either variable for prediction purposes. A theory order regressions using generalized product-moments. of generalized least-squares was developed by this author to Denition 2: Dene the generalized bivariate product- overcome this problem by minimizing the average generalized moment of order p = m + n as mean of the square deviations in both x and y variables [5]– N 1 m n [8]. For the resulting regression equation, one can solve for  = (xi  ) yi  (2) m;n N x y x in terms of y in order to predict the value of x: This theory i=1 was subsequently extended to multiple variables [9]. X
for whole numbers m and n. In this paper, the extension of the bivariate theory of gen- Theorem 3: (Explicit Bivariate Error Formula) Let p be an eralized least-squares to least-powers is begun. The bivariate even whole number and let F = E a= b : Then generalized least-powers problem of order p seeks a line which j y x minimizes the average generalized mean of the absolute pth p t power deviations between the data and the line. Unlike least- E = ( 1)s  br a  b (3) r; s; t r;s y x squares regressions which utilize second order moments of r+s+t=p   X
the data to construct the regression line, least-pth powers or regressions utilize moments of order p to construct the line. p t In the interest of generality, the denitions here are for- E = ( 1)s  br a  b + F r; s; t r;s y x mulated using arbitrary powers p. Nevertheless, the focus r+s+t=p;t=0   X 6
N. Greene is with the Department of Mathematics and Computer Science, where Kingsborough Community College, City University of New York, 2001 p p r p r Oriental Boulevard, Brooklyn, NY 11235 USA (phone: 718-368-5929; e-mail: F = ( 1) r;p rb : (4) r [email protected]). r=0 X  

ISSN: 1998-0140 352 INTERNATIONAL JOURNAL OF MATHEMATICAL MODELS AND METHODS IN APPLIED SCIENCES Volume 10, 2016

Proof: Assume p is even and omit absolute values. Begin For p = 4, with the error expression and manipulate as follows: 4 3 2 F = 4;0b 43;1b + 62;2b 41;3b + 0;4: (7) N 1 p E = (a + bxi yi) For p = 6, N i=1 F =  b6 6 b5 + 15 b4 20 b3 XN 6;0 5;1 4;2 3;3 1 p 2 = b (xi  ) yi  + a  b +152;4b 61;5b + 0;6: N x y y x i=1 (8) XN


1 p Theorem 5: (Bivariate OLP Regression) The OLPp y x = ( 1)s br j N r; s; t regression line y = a + bx is obtained by solving i=1 r+s+t=p   p X X s t r p r p (xi  ) yi  a  b r 1 x y y x F 0 (b) = ( 1) r r;p rb = 0 (9)  r s p r=1   = ( 1)
br
X r; s; t r+s+t=p for b and setting X   N 1 r s t a = y bx: (xi  ) yi  a  b  N x y y x (10) ( i=1 ) X

Example 6: For p = 2, which is least-squares, one solves p t = ( 1)s  br a  b : r; s; t r;s y x 0 =  b  : (11) r+s+t=p   2;0 1;1 X
Now separate out the terms with t = 0 and obtain For p = 4 one solves 3 2 s p t 0 =  b 3 b + 3 b  : (12) E = ( 1)  br a  b + F 4;0 3;1 2;2 1;3 r; s; t r;s y x r+s+t=p;t=0   For p = 6 one solves X 6
where 0 =  b5 5 b4 + 10 b3 6;0 5;1 4;2 p 2 F = ( 1)s  br 103;3b + 52;4b 1;5: r; s; 0 r;s r+s=p   (13) Xp Now that the analog of OLS, called OLP, has been de- p r p r = ( 1) r;p rb : scribed, the corresponding bivariate theory of generalized r r=0 least-powers can be described as well. X   Observe that applying the trinomial expansion theorem to the error expression E and then setting a = y bx produces the B. Bivariate Generalized Means and XMRp Notation same result F as rst setting a =  b and then applying y x The axioms of a generalized mean were stated by us the binomial expansion: previously [8], [9] drawing from the work of Mays [11] N and also from Chen [2]. They are stated here again for 1 p F = b (xi  ) yi  convenience. N x y i=1 Denition 7: A function M (x; y) denes a generalized X

N p mean for x; y > 0 if it satises Properties 1-5 below. If 1 p r r p r p r = b (xi  ) yi  ( 1) N r x y it satises Property 6 it is called a homogeneous generalized i=1 r=0   mean. The properties are: p X X
p r p = ( 1) 1. (Continuity) M (x; y) is continuous in each variable. r 2. (Monotonicity) M (x; y) is non-decreasing in each vari- r=0   X N able. 1 r p r r (xi  ) yi  b 3. (Symmetry) M (x; y) = M (y; x) :  N x y ( i=1 ) 4. (Identity) M (x; x) = x: p X
5. (Intermediacy) min (x; y) M (x; y) max (x; y) : p r p r   = ( 1) r;p rb : 6. (Homogeneity) M (tx; ty) = tM (x; y) for all t > 0: r r=0 X   All known means are included in this denition. All the means discussed in this paper are homogeneous. The Example 4: For p = 2, which is least-squares, generalized mean of any two generalized means is itself a generalized mean. F =  b2 2 b +  : (5) 2;0 1;1 0;2 XMRp notation is used here to name generalized regres- In more familiar notation this is sions: if `X' is the letter used to denote a given generalized mean, then XMRp is the corresponding generalized least-pth 2 2 2 F =  b 2xyb +  : (6) power regression. XMR2 is the same thing as XMR without a x y

ISSN: 1998-0140 353 INTERNATIONAL JOURNAL OF MATHEMATICAL MODELS AND METHODS IN APPLIED SCIENCES Volume 10, 2016 subscript. For example, `G' is the letter usually used to denote D. How to Find the Regression Coefcients the geometric mean and GMR is least-pth power geometric p The fundamental practical question of bivariate generalized mean regression. AMR is least-pth power arithmetic mean p least-powers regression is how to nd the coefcients a and regression. The generalization of orthogonal least-squares re- b for the regression line y = a + bx. The case of least- gression to least-powers is HMR since orthogonal regression p even-power regressions can be solved analytically, analogous is the same as harmonic mean regression. to how generalized least-squares regressions are solved. For p even, to nd the regression coefcients a and b; take the rst order partial derivatives of the error function E and E and C. The Two Generalized Least-Powers Problems and the a b set them equal to zero. Solving Ea = 0 yields a = y bx. Equivalence Theorem @E Solving Eb = @b = 0 and then setting a = y bx is The general symmetric least-powers problem is stated as equivalent to setting a =  b rst and then solving y x follows. dF = @E = 0: The latter procedure is employed db @b a=y bx Denition 8: (The General Symmetric Least-Powers Prob- here because it is simpler. lem) Values of a and b are sought which minimize an error Theorem 11: (Solving for the Generalized Regression Co- function dened by efcients) For p even, the generalized least-powers slope b is found by solving: N p 1 p a 1 E = M a + bxi yi ; + xi yi (14) d N j j b b i=1   g (b) F (b) = 0 (18) X db f g where M (x; y) is any generalized mean. where Denition 9: (The General Weighted Ordinary Least- p p r p r Powers Problem) Values of a and b are sought which minimize F = ( 1) r;p rb (19) r r=0 an error function dened by X  

N and the y-intercept a is given by 1 p E = g (b) a + bxi yi : (15)
N j j a = y bx: (20) i=1 X where g (b) is a positive even function that is non-decreasing for b < 0 and non-increasing for b > 0. E. The Hessian Matrix The next theorem states that every generalized least-powers In order for the regression coefcients (a; b) to minimize regression problem is equivalent to a weighted ordinary least- the error function and be admissible, the Hessian matrix of powers problem with weight function g (b). second order partial derivatives must be positive denite when Theorem 10: Every general symmetric least-powers error evaluated at (a; b). The general Hessian matrix is calculated function can be written equivalently as next. As in the case of generalized least-squares, certain combinations of g and its rst and second partial derivatives N 1 p appear in the matrix. One combination is denoted here by E = g (b) a + bxi yi (16)
N j j J and another is denoted by G. They are called indicative i=1 X functions. where Denition 12: Dene the indicative functions 1 2 g (b) = M 1; : g00 (b) g0 (b) b p J (b) = 2 (21)   g (b) g (b) j j (17)   a 1 1 and Proof: Substitute +xi yi with (a + bxi yi) and b b b 2g (b) g (b) then use the homogeneity property: G (b) = 0 00 : (22) g (b) g0 (b) N p The two indicative functions are related by the equation 1 p a + bxi yi E = M a + bxi yi ; j j F =F = J=G. The latter differential equation can be solved N j j b p 0 i=1  j j  explicitly for g (b). One obtains [6] XN 1 p 1 1 = a + bxi yi M 1; : N j j b p g (b) = : (23) i=1   c + k exp G (b) db db X j j Dene Theorem 13: (HessianR matrix) R The Hessian
matrix H of 1 g (b) = M 1; ; second order partial derivatives of the error function given by b p  j j  H H H = 11 12 : (24) and factor g (b) outside of the summation. H21 H22  

ISSN: 1998-0140 354 INTERNATIONAL JOURNAL OF MATHEMATICAL MODELS AND METHODS IN APPLIED SCIENCES Volume 10, 2016 is computed explicitly as follows: Theorem 14: The Hessian matrix H is positive denite at the solution point (a; b) provided that det H > 0: H11 = Eaa = p (p 1) gFp 2 (25) Proof: H is positive denite provided that H11 > 0 and g0 det H > 0. In our case H11 = g (b) p (p 1) Fp 2 (b) > 0 H12 = H21 = Eab = Eba = pg Fp 1 + Fp0 1

g for all b. Therefore it sufces to evaluate the determinant   (26) numerically at the slope b.

H22 = Ebb = g (JF + F 00) : (27) III.REGRESSION EXAMPLES BASEDON KNOWN SPECIAL Alternatively, MEANS H = g (GF + F ) : (28) 22 0 00 A. Arithmetic Mean Regression (AMRp) Proof: Take second order partial derivatives of the error The arithmetic mean is given by and evaluate at the solution (a; b). Let E = EOLP for purposes 1 of this proof and assume p is even. A (x; y) = (x + y) : (29) 2 @2 This mean generates arithmetic mean regression AMRp. The H22 = 2 (gE) @b weight function is given by @ = (g E + gE ) @b 0 b 1 1 g (b) = 1 + p : (30) = g E + 2g E + gE : 2 b 00 0 b bb  j j  g0 The general AMRp slope equation for p even is given by Since g0E + gEb = 0 at the solution, substitute Eb = g E p+1 into the middle term, simplify, and obtain b + b F 0 (b) pF (b) = 0: (31) 2 g00 g0 The AMR2 slope equation
is H22 = g 2 E + Ebb g g ! ! 0 =  b4  b3 +  b  : (32)   2;0 1;1 1;1 0;2 which is the rst form of the Hessian. Now substitute E = The AMR4 slope equation is g g Eb and obtain the second form 8 7 6 5 0 0 = 4;0b 33;1b + 32;2b 1;3b 3 2 2g0 g00 +3;1b 32;2b + 31;3b 0;4: H22 = g Eb + Ebb : g g  0   (33) As before, upon substituting for a, E = F , E = F and b 0 The AMR6 slope equation is E = F . For H ; bb 00 11 12 11 10 9 0 = 6;0b 55;1b + 104;2b 103;3b @2 +5 b8  b7 +  b5 5 b4 H11 = 2 (gE) 2;4 1;5 5;1 4;2 @a 3 2 2 N +103;3b 102;4b + 51;5b 0;6: (34) @ 1 p = g (a + bxi yi) @a2
N i=1 ! B. Geometric Mean Regression (GMRp) X N 1 p 2 The geometric mean is given by = g p (p 1) (a + bxi yi)

N 1=2 i=1 G (x; y) = (xy) : (35) X 1 N This mean generates geometric mean regression GMRp. The = g p (p 1) b (xi  ) yi 

N x y i=1 weight function is given by Xp 2
p=2 + a  b g (b) = b : (36) y x j j N 1

p 2 The general GMRp slope equation for p even is = g p (p 1) b (xi  ) yi 

N x y i=1 2bF 0 (b) pF (b) = 0: (37) X

= g p (p 1) Fp 2:

The GMR2 slope equation is For H12 = H21, 0 =  b2  : (38) 2;0 0;2 @2 H = (gE) The GMR4 slope equation is 12 @b@a N 4 3 0 = 4;0b 23;1b + 21;3b 0;4: (39) @ 1 p 1 = p g (a + bxi yi) @b
N The GMR6 slope equation is i=1 ! X 6 5 4 = p g0Fp 1 + gFp0 1 : 0 = 6;0b 45;1b + 54;2b 5 b2 + 4 b  : (40)
2;4 1;5 0;6

ISSN: 1998-0140 355 INTERNATIONAL JOURNAL OF MATHEMATICAL MODELS AND METHODS IN APPLIED SCIENCES Volume 10, 2016

C. Harmonic Mean (Orthogonal) Regression (HMRp) The OLP2 x y slope equation is j The harmonic mean is given by 0 =  b  : (53) 2xy 1;1 0;2 H (x; y) = : (41) x + y The OLP4 x y slope equation is j This mean generates harmonic mean regression HMRp. The 0 =  b3 3 b2 + 3 b  : (54) weight function is given by 3;1 2;2 1;3 0;4 2 g (b) = : (42) The OLP6 x y slope equation is 1 + b p j j j 5 4 3 The general HMRp slope equation for p even is 0 = 5;1b 54;2b + 103;3b 2 p p 1 102;4b + 51;5b 0;6: (55) (b + 1) F 0 (b) pb F (b) = 0: (43) The HMR2 slope equation is IV. REGRESSION EXAMPLES BASEDON GENERALIZED 2 0 = 1;1b + 2;0 0;2 b 1;1: (44) MEANS The HMR4 slope equation is
The generalized means in the next examples have free 6 5 4 parameters which can be used to parameterize these and other 0 = 3;1b 32;2b + 31;3b known special cases. +   b3 3 b2 4;0 0;4 3;1 +3 b  : (45) 2;2 1;3
A. Weighted Arithmetic Mean Regression The HMR6 slope equation is 0 =  b10 5 b9 + 10 b8 10 b7 The weighted arithmetic mean with weight in [0; 1] is 5;1 4;2 3;3 2;4 given by 6 5 4 +51;5b + 6;0 0;6 b 55;1b M (x; y) = (1 ) x + y (56) +10 b3 10 b2 + 5 b  : (46) 4;2 3;3
2;4 1;5 for x y. Harmonic mean regression is the same thing as orthogonal  regression. This is because of the Reciprocal Pythagorean The weight function corresponding to the weighted arith- Theorem [4], [12] which says that the diagonal deviation metic mean is between a data point and the regression line is half the p harmonic mean of the horizontal and vertical deviations. g (b) = (1 ) + b : (57) j j

For weighted AMRp; the general slope equation for p even D. Ordinary Least-Powers x y Regression (OLPp x y) j j is The selection mean given by p+1 (1 ) b + b F 0 (b) pF (b) = 0: (58) S (x; y) = x (47) x
The weighted AMR2; slope equation is generates OLPp y x regression. The selection mean given by j 4 3 0 = (1 ) 2;0b (1 ) 1;1b + 1;1b 0;2 (59) Sy (x; y) = y (48) generates OLPp x y regression. The weight function corre- The weighted AMR4; slope equation is j sponding to OLPp y x is given by j 8 7 0 = (1 ) 4;0b 3 (1 ) 3;1b g (b) = 1: (49) 6 5 +3 (1 ) 2;2b (1 ) 1;3b 3 2 The weight function corresponding to OLPp x y is given by +  b 3  b j 3;1 2;2 1 +3 1;3b 0;4: (60) g (b) = p : (50) b j j The weighted AMR6; slope equation is The general OLPp y x slope equation for p even is j 0 = (1 )  b12 5 (1 )  b11 F 0 (b) = 0 (51) 6;0 5;1 +10 (1 )  b10 10 (1 )  b9 The specic equations for p = 2; 4; and 6 were already 4;2 3;3 8 7 described. +5 (1 ) 2;4b (1 ) 1;5b 5 4 3 The general OLPp x y slope equation for p even is +  b 5  b + 10  b j 5;1 4;2 3;3 10  b2 + 5  b  (61) bF 0 (b) pF (b) = 0 (52) 2;4 1;5 0;6

ISSN: 1998-0140 356 INTERNATIONAL JOURNAL OF MATHEMATICAL MODELS AND METHODS IN APPLIED SCIENCES Volume 10, 2016

B. Weighted Geometric Mean Regression The PMR2;q slope equation is The weighted geometric mean with weight in [0; 1] is 2q+2 2q+1 0 = 2;0b 1;1b + 1;1b 0;2 (71) given by 1 M (x; y) = x y (62) The PMR4;q slope equation is 4q+4 4q+3 for x y. 0 = 4;0b 33;1b  4q+2 4q+1 The weight function corresponding to the weighted geomet- +32;2b b 1;3 ric mean is given by 3 2 +3;1b 32;2b + 31;3b 0;4: (72) p g (b) = b : (63) j j The PMR6;q slope equation is For weighted GMRp; the general slope equation for p even 6q+6 6q+5 6q+4 0 = 6;0b 55;1b + 104;2b is 6q+3 6q+2 6q+1 103;3b + 52;4b 1;5b bF 0 (b) p F (b) = 0: (64) 5 4 3 +5;1b 54;2b + 103;3b The weighted GMR slope equation is 2; 10 b2 + 5 b  : (73) 2;4 1;5 0;6 0 = (1 )  b2 + (2 1)  b  (65) 2;0 1;1 0;2 D. Regressions Based on Other Generalized Means The weighted GMR4; slope equation is As was done in our previous paper [8] for p = 2, regression 0 = (1 )  b4 (3 4 )  b3 4;0 3;1 formulas for p > 2 based on other generalized means can +3 (1 2 )  b2 (1 4 )  b  : be worked out. The Dietel-Gordon Mean of order r, the 2;2 1;3 0;4 (66) Stolarsky mean of order s; the two-parameter Stolarsky mean of order (r; s), the Gini mean of order t, the two-parameter The weighted GMR6; slope equation is Gini mean of order (r; s) are alternative generalized means which parameterize the known specic cases. Details on these 0 = (1 )  b6 (5 6 )  b5 6;0 5;1 means and the corresponding references can be found in that +5 (2 3 )  b4 10 (1 2 )  b3 4;2 3;3 paper. We leave the detailed slope equations in these cases +5 (1 3 )  b2 (1 6 )  b  : for a future work in this series. 2;4 1;5 0;6 (67) V. EQUIVALENCE THEOREMS C. Power Mean Regression A. Solving for the Generalized Mean Parameter as a Function of the Slope The power mean of order q, for q = 0, is given by 6 For the case of weighted AMR, weighted GMR, and PMR 1=q 1 the free parameter in these cases can solved for explicitly M (x; y) = (xq + yq) (68) q 2 in terms of the slope. The result of doing this yields an   equivalence theorem. with M0 (x; y) = G (x; y), M (x; y) = min (x; y) and 1 Theorem 15: (Weighted Arithmetic Mean Equivalence The- M (x; y) = max (x; y). 1 orem) Let b be the slope of a generalized least-powers regres- Many other special means are specic cases as well: q = 1 sion line with p even. Then the line can be generated by an is the harmonic mean, q = 1 was the basis for squared 2 equivalent weighted arithmetic mean regression with weight harmonic mean regression (SHR), q = 1 was the basis for 2 given by square perimeter regression (SPR), q = 1 is the arithmetic mean and q = 2 is called the root-mean-square [5], [8]. bp+1F (b) = 0 (74) Many other special means are approximated well by (bp+1 b) F (b) + pF (b) 0 power means: M 1=3 (x; y) approximates the second log- with 2 1=3 arithmic mean L2 (x; y) and HG well, M1=3 (x; y) ; b = bOLP y x + ! bOLP x y bOLP y x (75) called the Lorentz mean, approximates the rst logarithmic j j j
mean L1 (x; y) well, and M2=3 (x; y) approximates both the and ! in [0; 1] :
Heronian mean N (x; y) and the identric mean I (x; y) well. Proof: Solve the weighted AMRp; slope equation for This is proven in our earlier paper [8]. . The weight function is given by Theorem 16: (Weighted Geometric Mean Equivalence The-

1=q orem) Let b be the slope of a generalized least-powers regres- 1 pq g (b) = 1 + b : (69) sion line with p even. Then the line can be generated by an 2 j j   equivalent weighted geometric mean regression with weight   given by The general PMRp;q slope equation for p even is bF 0 (b) pq = (76) (b + 1) bF 0 (b) pF (b) = 0: (70) pF (b)

ISSN: 1998-0140 357 INTERNATIONAL JOURNAL OF MATHEMATICAL MODELS AND METHODS IN APPLIED SCIENCES Volume 10, 2016 with or simply 2 b = bOLP y x + ! bOLP x y bOLP y x (77) F 00 (b) F (b) (F 0 (b)) = 0 j j j and ! in [0; 1] :
for b. Call the resulting value bEXT. To insure that the value Proof: Solve the weighted GMRp; slope equation for is a maximum, one must also verify that . Theorem 17: (Power Mean Equivalence Theorem) Let b be P 00 (bEXT) < 0: p the slope of a generalized least-powers regression line with The corresponding maximum value of P is P0 = P (bEXT). even. Then the line can be generated by an equivalent power Example 19: For p = 2 one obtains mean regression of order q with 2 2 2 2;0b 22;01;1b + 2;00;2 21;1 = 0 (81) 1 pF (b) bF 0 (b) q = ln = ln b: (78) p bF (b) which in covariance notation is
 0  4 2 2 2 2 2 Proof: Solve the weighted PMRp;q slope equation for q:  b 2 xyb +   2 = 0: (82) x x x y xy This can be solved explicitly to obtain the slope
of the extremal For b > 0, if the interval bOLP y x; bOLP x y does not j j line. It can also be expressed using covariances or standard contain 1, or for b < 0, if the interval bOLP x y; bOLP y x does not contain 1, then every regression line lyingj betweenj deviations and correlation coefcients as in the previous papers the two ordinary least-powers lines is generated by a power [7], [8]: mean of order q for some q in ( ; ) with the ordinary    2 least-powers lines corresponding to1q =1 . This was shown 1;1  2;0 0;2 1;1 1 b = (83) in detail for the case of least-squares [8]. q 2;0 2 2 2 xy xy xy B. The Exponential Equivalence Theorem and the Fundamen-  = 2 (84) tal Formula for Generalized Least-Powers Regression qx y y In the bivariate case of generalized least-squares it was =  1 2: (85) x  x shown [7], [8] that every weighted ordinary least-squares p regression line can be generated by an equivalent exponen- For p = 4 one obtains tially weighted regression with weight function g0 (b) = 2 6 5 0 = 4;0b 64;03;1b exp ( P b ) for in [0; 1]. This theorem generalizes to 0 + 122 + 3  b4 least-powers j regressionsj as well. 3;1 4;0 2;2 3 Theorem 18: (The Extremal Line) For exponentially 243;12;2 44;01;3 b
weighted ordinary least-powers regression with weight func- 3  182 b2 4;0 0;4 2;2
tion g (b) = exp ( P b ), the regression line generated by the 12  8  b maximum value of Pj isj called the extremal line. The slope 2;2 1;3 3;1
0;4 3  42 : (86) of the extremal line bEXT is computed by solving 2;2 0;4 1;3
2 For p = 6 one obtains F 00 (b) F (b) (F 0 (b)) = 0 (79)
0 = 2 b10 10  b9 and the maximum value of P , called P0, is computed by 6;0 6;0 5;1 + 302 + 15  b8 F 0 (bEXT) 5;1 6;0 4;2 P0 = sgn (bEXT) : (80) 7 F (bEXT) 120 5;14;2b
2 6 As the parameter P varies over the interval [0;P ] all 206;02;4 805;13;3 1504;2 b 0 exponentially weighted least-powers regressions are generated. 5 + 305;12;4 3004;23;3 + 186;0
1;5 b Proof: Consider the weight function g (b) = + 75  5  exp ( P b ). To nd the slope b, one must solve 4;2 2;4 6;0 0;6
j j 60  + 2002 b4 d 5;1 1;5 3;3 exp ( P b ) F (b) = 0 + 20  200  + 60  b3 db f j j g 5;1 0;6 3;
3 2;4 4;2 1;5 or 30  752 b2 4;2 0;6 2;4
+ 20  30  b exp ( P b )( P ) (sgn b) F (b) + exp ( P b ) F 0 (b) = 0 3;3 0;6 2;4
1;5 j j j j 5  62 : which is equivalent to writing 2;4 0;6 1;5
(87) F (b)
P (b) = sgn b 0 : Since one cannot solve these equations explicitly when p > F (b) 2 one solves the equations numerically for b instead. One To nd the maximum value of P one must solve selects the real root that shares the same sign as bOLP and is 2 greater than bOLP in absolute value. F 00 (b) F (b) (F 0 (b)) P 0 (b) = sgn b = 0 (F (b))2

ISSN: 1998-0140 358 INTERNATIONAL JOURNAL OF MATHEMATICAL MODELS AND METHODS IN APPLIED SCIENCES Volume 10, 2016 Theorem 20: (Exponential Equivalence Theorem) Dene 13:9352,  = 14:1250,  = 14:7292. The sixth order 3;1 4;0 the normalized exponential parameter = P=P0 so that product-moments are:  = 87:7956,  = 86:0802, 0;6 1;5 g0 (b) = exp ( P0 b ) for in [0; 1]. Then every weighted  = 84:8200,  = 83:9583,  = 83:6227,  = j j 2;4 3;3 4;2 5;1 generalized least-powers regression line with slope b is also 83:9063, 6;0 = 85:1823: generated by an equivalent exponentially weighted regression The generalized regression lines are plotted together with with normalized exponential parameter the extremal line thereby displaying the region containing all 1 F (b) admissible generalized regression lines. = sgn b 0 : (88) P0 F (b) p = 2 p = 4 p = 6

The case = 0 is OLPp y x regression. The case = 1 is 7 7 7 the extremal line. j 6 6 6 Proof: For an arbitrary weight function g (b), the slope b is found by solving d g (b) F (b) =db = 0 which is equivalent 5 5 5 f g to g0 (b) =g (b) = F 0 (b) =F (b) at the solution. However, 4 4 4 it is already known that for a xed slope b there exists a y y y constant P in [0;P0] such that P sgn b = F 0 (b) =F (b) : Thus 3 3 3 P = (sgn b) g0 (b) =g (b) and for every weight function g and slope b there is a corresponding value for the exponential 2 2 2 parameters P and = P=P0. 1 1 1 Since b always lies in the interval between bOLP and bEXT; 0 0 0 0 2 4 6 0 2 4 6 0 2 4 6 the fundamental formula of generalized least-powers regres- x x x sion follows. Theorem 21: (Fundamental Formula of Generalized Least- Powers Regression) Every weighted generalized least-powers regression line y = a + bx has the form The equation of each line is presented along with the exponential parameters and , the weighted arithmetic mean b = bOLP +  (bEXT bOLP) (89) parameter , and the weighted geometric mean parameter 1 . In all cases one uses = (sgn b) F 0 (b) =F (b) and a = y bx (90) P0  = (b bOLP) = (bEXT bOLP) : According to the exponential for some  =  ( ) in [0; 1]. equivalence theorem, all regression lines are generated for Once the slope b of a regression line is known, the corre- p = 2 by g0 (b) = exp ( 2:6584 b ), for p = 4 by sponding value of  can be determined numerically using j j g0 (b) = exp ( 4:3714 b ), and for p = 6 by g0 (b) = j j b bOLP exp ( 6:4294 b ).  = : (91) j j bEXT bOLP The function = () can be determined explicitly by p = 2 += bxay g l a b composing = (b) with b = b (). The result is OLP2 | xy y -= 8571.04762.5 x .0 0000 .0 0000 .0 0000 .0 0000 HMR y -= 9304.06593.5 x .0 3752 .0 1947 .0 4283 .0 4640 1 F 0 (bOLP +  (bEXT bOLP)) 2 y -= 9361.06735.5 x = sgn b : (92) GMR 2 .0 4019 .0 2098 .0 4670 .0 5000 P0 F (bOLP +  (bEXT bOLP)) AMR y -= 9409.06855.5 x .0 4241 .0 2226 .0 5000 .0 5304 2 OLP | yx y -= 0222.18889.5 x .0 7360 .0 4389 .1 0000 .1 0000 For p = 2 only, the parameters and  are related by the 2

y -= 2333.14166.6 x _ _ 1 1 1 EXT2 .1 0000 .1 0000 simple formulas = sin 2 tan  and  = tan 2 sin [8]. p = 4 += bxay g l a b

OLP4 | xy y -= 7864.02993.5 x .0 0000 .0 0000 .0 0000 .0 0000 HMR y -= 8515.04622.5 x .0 3703 .0 0920 .0 3446 VI.NUMERICAL EXAMPLE 4 .0 2166 GMR 4 y -= .05750.5 8967x .0 5102 .0 1557 .0 3926 .0 5000

This section revisits an example explored in the previous AMR 4 y -= 9276.06523.5 x .0 5668 .0 1993 .0 5000 .0 5746 OLP | yx y .6 2767 -= 1774.1 x .0 7772 .0 5519 .1 0000 .1 0000 work. Regressions corresponding to OLPp, HMRp, GMRp, 4

EXT y -= 4948.10703.7 x .1 0000 .1 0000 _ _ AMRp and the extremal line are computed for p = 2; 4; 4 and 6. The corresponding effective weighted arithmetic and p = 6 += bxay g l a b geometric mean parameters and are computed. The OLP6 | xy y -= 7562.02239.5 x .0 0000 .0 0000 .0 0000 .0 0000 y -= 7902.03088.5 x effective exponential parameters and  are also computed. HMR 6 .0 2312 .0 0407 .0 0559 .0 1958 GMR y -= 9183.06291.5 x .0 5081 .0 1939 .0 3749 .0 5000 Example 22: This example appears in our previous papers 6 AMR y -= 9655.07471.5 x .0 5340 .0 2504 .0 5000 .0 5524 and is originally from Martin [10]. Six data values are given: 6 OLP | yx y .6 4719 -= 2554.1 x .0 7433 .0 5973 .1 0000 .1 0000 (0; 6), (1; 4), (2; 3), (3; 4), (4; 2), and (5; 1). The reader can 6 EXT6 y -= 5921.13135.7 x .1 0000 .1 0000 _ _ verify that  = 0:9157,  = 2:5000,  = 3:3333, x = x y 1:70781,: and5986.y = The second order product-moments are: 0;2 = 2:5556, It is readily observed that for p = 2, the slope values lie in 1;1 = 2:5000, 2;0 = 2:9167: The fourth order product- the interval [ 1:2333; 0:8571] with a range of 0:3762. For moments are:  = 13:9630,  = 13:8333,  = p = 4, the slope values lie in the interval [ 1:4948; 0:7864] 0;4 1;3 2;2

ISSN: 1998-0140 359 INTERNATIONAL JOURNAL OF MATHEMATICAL MODELS AND METHODS IN APPLIED SCIENCES Volume 10, 2016 with a range of 0:7084. The range for p = 4 is approximately given by a = y bx. This is referred to here as the 1.9 times as long as the range for p = 2: For p = 6, the slope fundamental formula of generalized least-powers regression, values lie in the interval [ 1:5921; 0:7562] with a range of since it characterizes all possible regression lines in a simple 0:8359. The range for p = 6 is approximately 2.2 times as way. long as the range for p = 2: A simple numerical example shows generalized least- For p = 2, the interval between the two OLP slopes is powers regressions performing comparably to generalized [ 1:0222; 0:8571] with a range of 0:1651. For p = 4, the least-squares but with a wider range of slope values. The ap- interval between the two OLP slopes is [ 1:1774; 0:7864] plication of bivariate generalized least-powers to non-normally with a range of 0:3910. The range for p = 4 is approximately distributed data and the potential advantage of these methods 2.4 times as long as the range for p = 2: For p = 6, the over generalized least-squares is a subject of the next paper in interval between the two OLP slopes is [ 1:2554; 0:7562] this series. The extension of this theory to multiple variables with a range of 0:4992. The range for p = 6 is approximately is also a subject of the next paper in this series. 3.0 times as long as the range for p = 2: REFERENCES VII.SUMMARY [1] G. Arbia, "Least Quartic Regression Criterion with Application to Least-powers regressions minimizing the average gener- Finance," arXiv:1403.4171 [q-n.ST], 2014. [2] H. Chen. "Means Generated by an Integral." Mathematics Magazine, alized mean of the absolute pth power deviations between vol. 82, p. 370, Dec. 2005. the data and the regression line are described in this paper. [3] S. C. Ehrenberg. "Deriving the Least-Squares Regression Equation." The Particular attention is paid to the case of p even, since American Statistician, vol. 37, p. 232, Aug. 1983. [4] V. Ferlini, Mathematics without (many) words, College Math Journal, this case admits analytic solution methods for the regression No. 33, p.170, 2002. coefcients. Ordinary least-squares regression generalizes to [5] N. Greene, "Generalized Least-Squares Regressions I: Efcient Deriva- ordinary least-powers regression. The case p = 2 corresponds tions," in Proceedings of the 1st International Conference on Compu- tational Science and Engineering (CSE'13), Valencia, Spain, 2013, pp. to the generalized least-squares regressions of our previous 145-158. works. The specic cases of arithmetic, geometric and har- [6] N. Greene, "Generalized Least-Squares Regressions II: Theory and monic mean (orthogonal) regression are worked out in detail Classication," in Proceedings of the 1st International Conference on Computational Science and Engineering (CSE '13), Valencia, Spain, for the case of p = 2, 4 and 6. 2013, pp. 159-166. Regressions based on weighted arithmetic means of order [7] N. Greene, "Generalized Least-Squares Regressions III: Further Theory and weighted geometric means of order are also worked out. and Classication," in Proceedings of the 5th International Conference on Applied Mathematics and Informatics (AMATHI '14), Cambridge, The weights and continuously parameterize all generalized MA, 2014, pp. 34-38. regression lines lying between the two ordinary least-powers [8] N. Greene, "Generalized Least-Squares Regressions IV: Theory and lines. Power mean regression of order q has xed values Classication Using Generalized Means," in Mathematics and Comput- ers in Science and Industry, Varna, Bulgaria, 2014, pp. 19-35. of q corresponding to many known special means and offers [9] N. Greene, "Generalized Least-Squares Regressions V: Multiple Vari- another way to parameterize the generalized mean regressions ables," in New Developments in Pure and Applied Mathematics, Vienna, previously described. Austria, 2015, pp. 17-25. [10] S. B. Martin, Less than the Least: An Alternative Method of Least- Every generalized mean regression with error function given Squares Linear Regression, Undergraduate Honors Thesis, Department by of Mathematics, McMurry University, Abilene, Texas, 1998. [11] M. E. Mays. "Functions Which Parametrize Means." American Mathe- N p matical Monthly, vol 90, pp. 677-683, 1983. 1 p a 1 E = M a + bxi yi ; + xi yi (93) [12] R. B. Nelson. Proof Without Words: A Reciprocal Pythagorean Theo- N j j b b i=1   rem, Mathematics Magazine, Vol. 82, No. 5, p. 370, Dec. 2009. X [13] P. A. Samuelson, A Note on Alternative Regressions, Econometrica, Vol. is equivalent to a weighted ordinary least-powers regression 10, No. 1, pp. 80-83, Jan. 1942. with error function [14] R. Taagepera, Making Social Sciences More Scientic: The Need for Predictive Models, Oxford University Press, New York, 2008. N 1 p E = g (b) a + bxi yi (94)
N j j i=1 X and weight function 1 g (b) = M 1; (95) b p  j j  where M (x; y) is any generalized mean. The exponential equivalence theorem states that every weighted ordinary least-powers regression line can be gen- erated by an equivalent exponentially weighted regression with weight function g0 (b) = exp ( P0 b ) for in [0; 1]. j j The case = 0 corresponds to OLPp y x and the case = 1 corresponds to the extremal line.j It follows that every generalized least-powers line has slope given by b = bOLP +  (bEXT bOLP) for  =  ( ) in [0; 1] and y-intercept

ISSN: 1998-0140 360