Quantile Regression

QUANTILE REGRESSION AN INTRODUCTION ROGER KOENKER AND KEVIN F. HALLOCK UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN Abstract. Quantile regression as intro duced in Ko enker and Bassett 1978 may b e viewed as a natural extension of classical least squares estimation of conditional mean mo dels to the estimation of an ensemble of mo dels for conditional quantile functions. The central sp ecial case is the median regression estimator that mini- mizes a sum of absolute errors. The remaining conditional quantile functions are estimated by minimizing an asymmetrically weighted sum of absolute errors. Taken together the ensemble of estimated conditional quantile functions o ers a much more complete view of the e ect of covariates on the lo cation, scale and shap e of the distribution of the resp onse variable. This essay provides a brief tutorial intro duction to quantile regression metho ds, illustrating their application in several settings. Practical asp ects of computation, inference, and interpretation are discussed and suggestions for further reading are provided. 1. Introduction In the classical mythology of least-squares regression the conditional mean function, the function that describ es how the mean of y changes with the vector of covariates x, is almost all weneedtoknow ab out the relationship b etween y and x. In the resilient terminology of Frisch 1934 and Ko opmans 1937 it is \the `systematic comp onent' or `true value'," around which y uctuates due to an \erratic comp onent" or \accidental error." The crucial, and convenient, thing ab out this view is that the error is assumed to have precisely the same distribution whatever values maybetaken by the comp onents of the vector x.We refer to this as a pure lo cation shift mo del since it assumes that x a ects only the lo cation of the conditional distribution of y , not its scale, or any other asp ect of its distributional shap e. If this is the case, we can b e fully satis ed with an estimated mo del of the conditional mean function, supplemented p erhaps by an estimate of the conditional disp ersion of y around its mean. Version: Decemb er 28, 2000. Wewould like to thank Gib Bassett, John DiNardo, Olga Geling, and StevePortnoy for helpful comments on earlier drafts. This pap er has b een prepared for the Journal of Economic Perspectives \Symp osium on Econometric To ols". The researchwas partially supp orted by NSF grant SBR-9911184. 1 2 Quantile Regression When we add the further requirement that the errors are Gaussian, least-squares metho ds deliver the maximum likeliho o d estimates of the conditional mean function and achievea well-publicized optimality. Indeed, Gauss seems to have \discovered" the Gaussian densityasan ex post rationalization for the optimality of least-squares metho ds. But we will argue that there is more to econometric life than is dreamt of in this philosophy of the lo cation shift mo del. Covariates may in uence the conditional distribution of the resp onse in myriad other ways: expanding its disp ersion as in traditional mo dels of heteroscedasticity, stretching one tail of the distribution, compressing the other tail, and even inducing multimo dality. Explicit investigation of these e ects via quantile regression can provide a more nuanced view of the sto chastic relationship b etween variables, and therefore a more informative empirical analysis. The remainder of the pap er is organized as follows. Section 2 brie y explains how the ordinary quantiles, and consequently the regression quantiles, may b e de ned as the solution to a simple minimization of a weighted sum of absolute residuals. We then illustrate the technique in the classical bivariate setting of Ernst Engel's 1857 original fo o d exp enditure study. In Section 3 wesketch the outline of a more ambitious application to the analysis of infant birthweights in the U.S. In Section 4we o er a brief review of recent empirical applications of quantile regression in economics. Section 5 contains some practical guidance on matters of computation, inference, and software. Section 6 o ers several thumbnail sketches of \what can go wrong" and some p ossible remedies, and Section 7 concludes. 2. Whatisit? Wesay that a student scores at the th quantile of a standardized exam if he p erforms b etter than the prop ortion , of the reference group of students, and worse than the prop ortion 1 . Thus, half of students p erform b etter than the median student, and half p erform worse. Similarly, the quartiles divide the p opulation into four segments with equal prop ortions of the reference p opulation in each segment. The quintiles divide the p opulation into 5 parts; the deciles into 10 parts. The quantiles, or p ercentiles, or o ccasionally fractiles, refer to the general case. Quantile regression seeks to extend these ideas to the estimation of conditional quantile functions {models in which quantiles of the conditional distribution of the resp onse variable are expressed as functions of observed covariates. To accomplish this task weneeda newwayto de ne the quantiles. Quantiles, and their dual identities, the ranks, seem inseparably linked to the op er- ations of ordering and sorting that are generally used to de ne them. So it may come as a mild surprise to observethatwe can de ne the quantiles, and the ranks, through a simple alternative exp edient { as an optimization problem. Just as we can de ne the sample mean as the solution to the problem of minimizing a sum of squared residuals, we can de ne the median as the solution to the problem of minimizing a sum of absolute residuals. What ab out the other quantiles? If the symmetric absolute value Roger Koenker and Kevin F. Hallock 3 function yields the median, mayb e we can simply tilt the absolute value to pro duce the other quantiles. This \pinball logic" suggests solving X min 2.1 y i 2< where the function is illustrated in Figure 2.1. To see that this problem yields the sample quantiles as its solutions, it is only necessary to compute the directional derivative of the ob jective function with resp ect to ,taken from the left and from 1 the right. ρτ (u) τ−1 τ Figure 2.1. Quantile Regression Function Having succeeded in de ning the unconditional quantiles as an optimization problem, it is easy to de ne conditional quantiles in an analogous fashion. Least squares regression o ers a mo del for how to pro ceed. If, presented with a random sample fy ;y ;::: ;y g,wesolve 1 2 n n X 2 2.2 min y ; i 2< i=1 1 1 Consider the median case with equal to : the directional derivatives are simply one-half 2 the sum of the signs of the residuals, y , with zero residuals counted as +1 from the rightand i ^ ^ as 1 from the left. If is taken to b e a value such that half the observations lie ab ove and half lie b elow, then b oth directional derivatives will b e p ositive, and since the ob jective function is ^ increasing in either direction must b e a lo cal minimum. But the ob jective function is a sum of convex functions and hence convex, so the lo cal minimum is also a global one. Such a solution is a median by our original de nition. The same argumentworks for the other quantiles, but nowwe have asymmetric weightingofthenumb er of observations with p ositive and negative residuals and ^ this leads to solutions corresp onding to the th quantiles. 4 Quantile Regression · · · · · · · · · · · · · · · · · · · · · · · · ·· ·· ·· · · ··· · ·· · · · · ········ · · Food Expenditure · ·· · · ········ · · · · ·· ········· ·· ·· · ············ ··· ····· ······ · ············ ·· · 500 1000········· 1500········ · 2000 ················· ·· ·········· · ······ 1000 2000 3000 4000 5000 Household Income Figure 2.2. Engel Curves for Fo o d: This gure plots data taken from Ernst Engel's 1857 study of the dep endence of households' fo o d ex- p enditure on household income. we obtain the sample mean, an estimate of the unconditional p opulation mean, EY . If wenow replace the scalar by a parametric function x; and solve n X 2 2.3 y x ; min i i p 2< i=1 we obtain an estimate of the conditional exp ectation function E Y jx. In quantile regression we pro ceed in exactly the same way.To obtain an estimate of the conditional median function, we simply replace the scalar in 2.1 by the 1 parametric function x ; and set to .Variants of this idea were prop osed in i 2 the mid 18th century by Boscovich, and subsequently investigated by Laplace and Edgeworth, among others. To obtain estimates of the other conditional quantile functions we simply replace absolute values by , and solve X 2.4 y x ; min i i p 2< ^ The resulting minimization problem, when x; is formulated as a linear function of parameters, can b e solved very eciently by linear programming metho ds. We will defer further discussion of computational asp ects of quantile regression to Section 5. Roger Koenker and Kevin F. Hallock 5 To illustrate the basic ideas we brie y reconsider a classical empirical application, Ernst Engel's 1857 analysis of the relationship b etween household fo o d exp enditure and household income. In Figure 2.2 we plot the data taken from 235 Europ ean working class households. Sup erimp osed on the plot are seven estimated quantile regression lines corresp onding to the quantiles 2f:05;:1;:25;:5;:75;:9;:95g. The median = :5 t is indicated by the dashed line; the least squares t is plotted as the dotted line.

Load more