Effects of Skewness and Kurtosis on Model Selection Criteria
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Higher-Order Asymptotics
Higher-Order Asymptotics Todd Kuffner Washington University in St. Louis WHOA-PSI 2016 1 / 113 First- and Higher-Order Asymptotics Classical Asymptotics in Statistics: available sample size n ! 1 First-Order Asymptotic Theory: asymptotic statements that are correct to order O(n−1=2) Higher-Order Asymptotics: refinements to first-order results 1st order 2nd order 3rd order kth order error O(n−1=2) O(n−1) O(n−3=2) O(n−k=2) or or or or o(1) o(n−1=2) o(n−1) o(n−(k−1)=2) Why would anyone care? deeper understanding more accurate inference compare different approaches (which agree to first order) 2 / 113 Points of Emphasis Convergence pointwise or uniform? Error absolute or relative? Deviation region moderate or large? 3 / 113 Common Goals Refinements for better small-sample performance Example Edgeworth expansion (absolute error) Example Barndorff-Nielsen’s R∗ Accurate Approximation Example saddlepoint methods (relative error) Example Laplace approximation Comparative Asymptotics Example probability matching priors Example conditional vs. unconditional frequentist inference Example comparing analytic and bootstrap procedures Deeper Understanding Example sources of inaccuracy in first-order theory Example nuisance parameter effects 4 / 113 Is this relevant for high-dimensional statistical models? The Classical asymptotic regime is when the parameter dimension p is fixed and the available sample size n ! 1. What if p < n or p is close to n? 1. Find a meaningful non-asymptotic analysis of the statistical procedure which works for any n or p (concentration inequalities) 2. Allow both n ! 1 and p ! 1. 5 / 113 Some First-Order Theory Univariate (classical) CLT: Assume X1;X2;::: are i.i.d. -
4.2 Variance and Covariance
4.2 Variance and Covariance The most important measure of variability of a random variable X is obtained by letting g(X) = (X − µ)2 then E[g(X)] gives a measure of the variability of the distribution of X. 1 Definition 4.3 Let X be a random variable with probability distribution f(x) and mean µ. The variance of X is σ 2 = E[(X − µ)2 ] = ∑(x − µ)2 f (x) x if X is discrete, and ∞ σ 2 = E[(X − µ)2 ] = ∫ (x − µ)2 f (x)dx if X is continuous. −∞ ¾ The positive square root of the variance, σ, is called the standard deviation of X. ¾ The quantity x - µ is called the deviation of an observation, x, from its mean. 2 Note σ2≥0 When the standard deviation of a random variable is small, we expect most of the values of X to be grouped around mean. We often use standard deviations to compare two or more distributions that have the same unit measurements. 3 Example 4.8 Page97 Let the random variable X represent the number of automobiles that are used for official business purposes on any given workday. The probability distributions of X for two companies are given below: x 12 3 Company A: f(x) 0.3 0.4 0.3 x 01234 Company B: f(x) 0.2 0.1 0.3 0.3 0.1 Find the variances of X for the two companies. Solution: µ = 2 µ A = (1)(0.3) + (2)(0.4) + (3)(0.3) = 2 B 2 2 2 2 σ A = (1− 2) (0.3) + (2 − 2) (0.4) + (3− 2) (0.3) = 0.6 2 2 4 σ B > σ A 2 2 4 σ B = ∑(x − 2) f (x) =1.6 x=0 Theorem 4.2 The variance of a random variable X is σ 2 = E(X 2 ) − µ 2. -
On the Efficiency and Consistency of Likelihood Estimation in Multivariate
On the efficiency and consistency of likelihood estimation in multivariate conditionally heteroskedastic dynamic regression models∗ Gabriele Fiorentini Università di Firenze and RCEA, Viale Morgagni 59, I-50134 Firenze, Italy <fi[email protected]fi.it> Enrique Sentana CEMFI, Casado del Alisal 5, E-28014 Madrid, Spain <sentana@cemfi.es> Revised: October 2010 Abstract We rank the efficiency of several likelihood-based parametric and semiparametric estima- tors of conditional mean and variance parameters in multivariate dynamic models with po- tentially asymmetric and leptokurtic strong white noise innovations. We detailedly study the elliptical case, and show that Gaussian pseudo maximum likelihood estimators are inefficient except under normality. We provide consistency conditions for distributionally misspecified maximum likelihood estimators, and show that they coincide with the partial adaptivity con- ditions of semiparametric procedures. We propose Hausman tests that compare Gaussian pseudo maximum likelihood estimators with more efficient but less robust competitors. Fi- nally, we provide finite sample results through Monte Carlo simulations. Keywords: Adaptivity, Elliptical Distributions, Hausman tests, Semiparametric Esti- mators. JEL: C13, C14, C12, C51, C52 ∗We would like to thank Dante Amengual, Manuel Arellano, Nour Meddahi, Javier Mencía, Olivier Scaillet, Paolo Zaffaroni, participants at the European Meeting of the Econometric Society (Stockholm, 2003), the Sympo- sium on Economic Analysis (Seville, 2003), the CAF Conference on Multivariate Modelling in Finance and Risk Management (Sandbjerg, 2006), the Second Italian Congress on Econometrics and Empirical Economics (Rimini, 2007), as well as audiences at AUEB, Bocconi, Cass Business School, CEMFI, CREST, EUI, Florence, NYU, RCEA, Roma La Sapienza and Queen Mary for useful comments and suggestions. Of course, the usual caveat applies. -
The Method of Maximum Likelihood for Simple Linear Regression
08:48 Saturday 19th September, 2015 See updates and corrections at http://www.stat.cmu.edu/~cshalizi/mreg/ Lecture 6: The Method of Maximum Likelihood for Simple Linear Regression 36-401, Fall 2015, Section B 17 September 2015 1 Recapitulation We introduced the method of maximum likelihood for simple linear regression in the notes for two lectures ago. Let's review. We start with the statistical model, which is the Gaussian-noise simple linear regression model, defined as follows: 1. The distribution of X is arbitrary (and perhaps X is even non-random). 2. If X = x, then Y = β0 + β1x + , for some constants (\coefficients", \parameters") β0 and β1, and some random noise variable . 3. ∼ N(0; σ2), and is independent of X. 4. is independent across observations. A consequence of these assumptions is that the response variable Y is indepen- dent across observations, conditional on the predictor X, i.e., Y1 and Y2 are independent given X1 and X2 (Exercise 1). As you'll recall, this is a special case of the simple linear regression model: the first two assumptions are the same, but we are now assuming much more about the noise variable : it's not just mean zero with constant variance, but it has a particular distribution (Gaussian), and everything we said was uncorrelated before we now strengthen to independence1. Because of these stronger assumptions, the model tells us the conditional pdf 2 of Y for each x, p(yjX = x; β0; β1; σ ). (This notation separates the random variables from the parameters.) Given any data set (x1; y1); (x2; y2);::: (xn; yn), we can now write down the probability density, under the model, of seeing that data: n n (y −(β +β x ))2 Y 2 Y 1 − i 0 1 i p(yijxi; β0; β1; σ ) = p e 2σ2 2 i=1 i=1 2πσ 1See the notes for lecture 1 for a reminder, with an explicit example, of how uncorrelated random variables can nonetheless be strongly statistically dependent. -
Use of the Kurtosis Statistic in the Frequency Domain As an Aid In
lEEE JOURNALlEEE OF OCEANICENGINEERING, VOL. OE-9, NO. 2, APRIL 1984 85 Use of the Kurtosis Statistic in the FrequencyDomain as an Aid in Detecting Random Signals Absmact-Power spectral density estimation is often employed as a couldbe utilized in signal processing. The objective ofthis method for signal ,detection. For signals which occur randomly, a paper is to compare the PSD technique for signal processing frequency domain kurtosis estimate supplements the power spectral witha new methodwhich computes the frequency domain density estimate and, in some cases, can be.employed to detect their presence. This has been verified from experiments vith real data of kurtosis (FDK) [2] forthe real and imaginary parts of the randomly occurring signals. In order to better understand the detec- complex frequency components. Kurtosis is defined as a ratio tion of randomlyoccurring signals, sinusoidal and narrow-band of a fourth-order central moment to the square of a second- Gaussian signals are considered, which when modeled to represent a order central moment. fading or multipath environment, are received as nowGaussian in Using theNeyman-Pearson theory in thetime domain, terms of a frequency domain kurtosis estimate. Several fading and multipath propagation probability density distributions of practical Ferguson [3] , has shown that kurtosis is a locally optimum interestare considered, including Rayleigh and log-normal. The detectionstatistic under certain conditions. The reader is model is generalized to handle transient and frequency modulated referred to Ferguson'swork for the details; however, it can signals by taking into account the probability of the signal being in a be simply said thatit is concernedwith detecting outliers specific frequency range over the total data interval. -
Sampling Student's T Distribution – Use of the Inverse Cumulative
Sampling Student’s T distribution – use of the inverse cumulative distribution function William T. Shaw Department of Mathematics, King’s College, The Strand, London WC2R 2LS, UK With the current interest in copula methods, and fat-tailed or other non-normal distributions, it is appropriate to investigate technologies for managing marginal distributions of interest. We explore “Student’s” T distribution, survey its simulation, and present some new techniques for simulation. In particular, for a given real (not necessarily integer) value n of the number of degrees of freedom, −1 we give a pair of power series approximations for the inverse, Fn ,ofthe cumulative distribution function (CDF), Fn.Wealsogivesomesimpleandvery fast exact and iterative techniques for defining this function when n is an even −1 integer, based on the observation that for such cases the calculation of Fn amounts to the solution of a reduced-form polynomial equation of degree n − 1. We also explain the use of Cornish–Fisher expansions to define the inverse CDF as the composition of the inverse CDF for the normal case with a simple polynomial map. The methods presented are well adapted for use with copula and quasi-Monte-Carlo techniques. 1 Introduction There is much interest in many areas of financial modeling on the use of copulas to glue together marginal univariate distributions where there is no easy canonical multivariate distribution, or one wishes to have flexibility in the mechanism for combination. One of the more interesting marginal distributions is the “Student’s” T distribution. This statistical distribution was published by W. Gosset in 1908. -
The Smoothed Median and the Bootstrap
Biometrika (2001), 88, 2, pp. 519–534 © 2001 Biometrika Trust Printed in Great Britain The smoothed median and the bootstrap B B. M. BROWN School of Mathematics, University of South Australia, Adelaide, South Australia 5005, Australia [email protected] PETER HALL Centre for Mathematics & its Applications, Australian National University, Canberra, A.C.T . 0200, Australia [email protected] G. A. YOUNG Statistical L aboratory, University of Cambridge, Cambridge CB3 0WB, U.K. [email protected] S Even in one dimension the sample median exhibits very poor performance when used in conjunction with the bootstrap. For example, both the percentile-t bootstrap and the calibrated percentile method fail to give second-order accuracy when applied to the median. The situation is generally similar for other rank-based methods, particularly in more than one dimension. Some of these problems can be overcome by smoothing, but that usually requires explicit choice of the smoothing parameter. In the present paper we suggest a new, implicitly smoothed version of the k-variate sample median, based on a particularly smooth objective function. Our procedure preserves many features of the conventional median, such as robustness and high efficiency, in fact higher than for the conventional median, in the case of normal data. It is however substantially more amenable to application of the bootstrap. Focusing on the univariate case, we demonstrate these properties both theoretically and numerically. Some key words: Bootstrap; Calibrated percentile method; Median; Percentile-t; Rank methods; Smoothed median. 1. I Rank methods are based on simple combinatorial ideas of permutations and sign changes, which are attractive in applications far removed from the assumptions of normal linear model theory. -
A Note on Inference in a Bivariate Normal Distribution Model Jaya
A Note on Inference in a Bivariate Normal Distribution Model Jaya Bishwal and Edsel A. Peña Technical Report #2009-3 December 22, 2008 This material was based upon work partially supported by the National Science Foundation under Grant DMS-0635449 to the Statistical and Applied Mathematical Sciences Institute. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. Statistical and Applied Mathematical Sciences Institute PO Box 14006 Research Triangle Park, NC 27709-4006 www.samsi.info A Note on Inference in a Bivariate Normal Distribution Model Jaya Bishwal¤ Edsel A. Pena~ y December 22, 2008 Abstract Suppose observations are available on two variables Y and X and interest is on a parameter that is present in the marginal distribution of Y but not in the marginal distribution of X, and with X and Y dependent and possibly in the presence of other parameters which are nuisance. Could one gain more e±ciency in the point estimation (also, in hypothesis testing and interval estimation) about the parameter of interest by using the full data (both Y and X values) instead of just the Y values? Also, how should one measure the information provided by random observables or their distributions about the parameter of interest? We illustrate these issues using a simple bivariate normal distribution model. The ideas could have important implications in the context of multiple hypothesis testing or simultaneous estimation arising in the analysis of microarray data, or in the analysis of event time data especially those dealing with recurrent event data. -
Chapter 23 Flexible Budgets and Standard Cost Systems
Chapter 23 Flexible Budgets and Standard Cost Systems Review Questions 1. What is a variance? A variance is the difference between an actual amount and the budgeted amount. 2. Explain the difference between a favorable and an unfavorable variance. A variance is favorable if it increases operating income. For example, if actual revenue is greater than budgeted revenue or if actual expense is less than budgeted expense, then the variance is favorable. If the variance decreases operating income, the variance is unfavorable. For example, if actual revenue is less than budgeted revenue or if actual expense is greater than budgeted expense, the variance is unfavorable. 3. What is a static budget performance report? A static budget is a budget prepared for only one level of sales volume. A static budget performance report compares actual results to the expected results in the static budget and reports the differences (static budget variances). 4. How do flexible budgets differ from static budgets? A flexible budget is a budget prepared for various levels of sales volume within a relevant range. A static budget is prepared for only one level of sales volume—the expected number of units sold— and it doesn’t change after it is developed. 5. How is a flexible budget used? Because a flexible budget is prepared for various levels of sales volume within a relevant range, it provides the basis for preparing the flexible budget performance report and understanding the two components of the overall static budget variance (a static budget is prepared for only one level of sales volume, and a static budget performance report shows only the overall static budget variance). -
Random Variables Generation
Random Variables Generation Revised version of the slides based on the book Discrete-Event Simulation: a first course L.L. Leemis & S.K. Park Section(s) 6.1, 6.2, 7.1, 7.2 c 2006 Pearson Ed., Inc. 0-13-142917-5 Discrete-Event Simulation Random Variables Generation 1/80 Introduction Monte Carlo Simulators differ from Trace Driven simulators because of the use of Random Number Generators to represent the variability that affects the behaviors of real systems. Uniformly distributed random variables are the most elementary representations that we can use in Monte Carlo simulation, but are not enough to capture the complexity of real systems. We must thus devise methods for generating instances (variates) of arbitrary random variables Properly using uniform random numbers, it is possible to obtain this result. In the sequel we will first recall some basic properties of Discrete and Continuous random variables and then we will discuss several methods to obtain their variates Discrete-Event Simulation Random Variables Generation 2/80 Basic Probability Concepts Empirical Probability, derives from performing an experiment many times n and counting the number of occurrences na of an event A The relative frequency of occurrence of event is na/n The frequency theory of probability asserts thatA the relative frequency converges as n → ∞ n Pr( )= lim a A n→∞ n Axiomatic Probability is a formal, set-theoretic approach Mathematically construct the sample space and calculate the number of events A The two are complementary! Discrete-Event Simulation Random -
Comparison of the Estimation Efficiency of Regression
J. Stat. Appl. Pro. Lett. 6, No. 1, 11-20 (2019) 11 Journal of Statistics Applications & Probability Letters An International Journal http://dx.doi.org/10.18576/jsapl/060102 Comparison of the Estimation Efficiency of Regression Parameters Using the Bayesian Method and the Quantile Function Ismail Hassan Abdullatif Al-Sabri1,2 1 Department of Mathematics, College of Science and Arts (Muhayil Asir), King Khalid University, KSA 2 Department of Mathematics and Computer, Faculty of Science, Ibb University, Yemen Received: 3 Jul. 2018, Revised: 2 Sep. 2018, Accepted: 12 Sep. 2018 Published online: 1 Jan. 2019 Abstract: There are several classical as well as modern methods to predict model parameters. The modern methods include the Bayesian method and the quantile function method, which are both used in this study to estimate Multiple Linear Regression (MLR) parameters. Unlike the classical methods, the Bayesian method is based on the assumption that the model parameters are variable, rather than fixed, estimating the model parameters by the integration of prior information (prior distribution) with the sample information (posterior distribution) of the phenomenon under study. By contrast, the quantile function method aims to construct the error probability function using least squares and the inverse distribution function. The study investigated the efficiency of each of them, and found that the quantile function is more efficient than the Bayesian method, as the values of Theil’s Coefficient U and least squares of both methods 2 2 came to be U = 0.052 and ∑ei = 0.011, compared to U = 0.556 and ∑ei = 1.162, respectively. Keywords: Bayesian Method, Prior Distribution, Posterior Distribution, Quantile Function, Prediction Model 1 Introduction Multiple linear regression (MLR) analysis is an important statistical method for predicting the values of a dependent variable using the values of a set of independent variables. -
Statistical Models in R Some Examples
Statistical Models Statistical Models in R Some Examples Steven Buechler Department of Mathematics 276B Hurley Hall; 1-6233 Fall, 2007 Statistical Models Outline Statistical Models Linear Models in R Statistical Models Regression Regression analysis is the appropriate statistical method when the response variable and all explanatory variables are continuous. Here, we only discuss linear regression, the simplest and most common form. Remember that a statistical model attempts to approximate the response variable Y as a mathematical function of the explanatory variables X1;:::; Xn. This mathematical function may involve parameters. Regression analysis attempts to use sample data find the parameters that produce the best model Statistical Models Linear Models The simplest such model is a linear model with a unique explanatory variable, which takes the following form. y^ = a + bx: Here, y is the response variable vector, x the explanatory variable, y^ is the vector of fitted values and a (intercept) and b (slope) are real numbers. Plotting y versus x, this model represents a line through the points. For a given index i,y ^i = a + bxi approximates yi . Regression amounts to finding a and b that gives the best fit. Statistical Models Linear Model with 1 Explanatory Variable ● 10 ● ● ● ● ● 5 y ● ● ● ● ● ● y ● ● y−hat ● ● ● ● 0 ● ● ● ● x=2 0 1 2 3 4 5 x Statistical Models Plotting Commands for the record The plot was generated with test data xR, yR with: > plot(xR, yR, xlab = "x", ylab = "y") > abline(v = 2, lty = 2) > abline(a = -2, b = 2, col = "blue") > points(c(2), yR[9], pch = 16, col = "red") > points(c(2), c(2), pch = 16, col = "red") > text(2.5, -4, "x=2", cex = 1.5) > text(1.8, 3.9, "y", cex = 1.5) > text(2.5, 1.9, "y-hat", cex = 1.5) Statistical Models Linear Regression = Minimize RSS Least Squares Fit In linear regression the best fit is found by minimizing n n X 2 X 2 RSS = (yi − y^i ) = (yi − (a + bxi )) : i=1 i=1 This is a Calculus I problem.