DOCSLIB.ORG

The Folding Test of Unimodality Alban Siffer, Pierre-Alain Fouque, Alexandre Termier, Christine Largouët

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
The Folding Test of Unimodality Alban Siffer, Pierre-Alain Fouque, Alexandre Termier, Christine Largouët Are your data gathered? The Folding Test of Unimodality Alban Siffer, Pierre-Alain Fouque, Alexandre Termier, Christine Largouët To cite this version: Alban Siffer, Pierre-Alain Fouque, Alexandre Termier, Christine Largouët. Are your data gath- ered? The Folding Test of Unimodality. KDD 2018 - 24th ACM SIGKDD International Confer- ence on Knowledge Discovery & Data Minin, Aug 2018, London, United Kingdom. pp.2210-2218, 10.1145/3219819.3219994. hal-01951676 HAL Id: hal-01951676 https://hal.archives-ouvertes.fr/hal-01951676 Submitted on 11 Dec 2018 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Are your data gathered? The Folding Test of Unimodality A. Siffer P. A. Fouque Univ. Rennes, Inria, CNRS, IRISA, Amossys Univ. Rennes, CNRS, IRISA, IUF [email protected] [email protected] A. Termier C. Largouet Univ. Rennes, Inria, CNRS, IRISA Univ. Rennes, Inria, CNRS, IRISA, agrocampus ouest [email protected] [email protected] ABSTRACT Unimodality tests exist in the literature (for example the dip Understanding data distributions is one of the most fundamental test [13] or the Silverman test [22]), however they are restricted to research topic in data analysis. The literature provides a great deal one dimensional data. Up to now, there exists no purely statistical of powerful statistical learning algorithms to gain knowledge on unimodality test able to tackle multi-dimensional data without dis- the underlying distribution given multivariate observations. We are tribution assumption. One may be tempted to use unidimensional likely to find out a dependence between features, the appearance unimodality tests on each dimension of multidimensional data (in a of clusters or the presence of outliers. Before such deep investi- way similar to [17]), however the simple example of Figure 1 shows gations, we propose the folding test of unimodality. As a simple the limits of such an approach. On the bottom left side, one can see statistical description, it allows to detect whether data are gathered that the bidimensional data is bimodal. However, its projection of or not (unimodal or multimodal). To the best of our knowledge, the X (top) and Y (right) axis are perfectly unimodal, preventing this is the first multivariate and purely statistical unimodality test. to detect the multimodality of that dataset with unidimensional It makes no distribution assumption and relies only on a straight- unimodality tests. forward p−value. Through real world data experiments, we show its relevance and how it could be useful for clustering. 100 CCS CONCEPTS • Mathematics of computing → Multivariate statistics; Non- 50 parametric statistics; Exploratory data analysis; 0 8 KEYWORDS 6 Unimodality test, Multivariate statistics 4 2 1 INTRODUCTION 0 Given a multidimensional dataset of numerical attributes, an impor- 2 tant question is to understand the “grouping behavior” of the data 4 points. This question is traditionally answered by clustering algo- 4 2 0 2 4 6 8 0 50 100 rithms. “Grouping behavior” being an ill-defined concept, clustering Figure 1: Unidimensional test fails in 2D algorithms answer a more precise question : they determine groups (clusters) of data points that are similar, while being different from the data points of the other groups. Sometimes, one may not need such a detailed result : a more basic “grouping behavior” question Our main contribution is to propose the first statistical unimodal- can be to determine if all the data points make one single, coherent ity test designed for multidimensional data. Our approach relies group or not. on a novel descriptive statistics for (multidimensional) data, that In statistical terms, this is formulated as determining if the distri- provides a measure of its unimodality character (level of unimodal- bution of the data points is unimodal or not. Per se, unimodality test ity). The proposed approach keeps the same elegant properties as is a first information about the data, as it tells that the data points well-known unimodality tests like dip: it is independant from the cannot be obviously separated into distinct groups. In a medical distribution of the data and has only one parameter (a p−value experiment, this would for example tell that the patients all reacted threshold to tune the confidence in the results). We provide an in a similar way, and that no group of patients deviated significantly algorithm for computing our test and show that it can be adapted from the others. Unimodality can then be seen as a building block to the streaming context through an incremental version. for clustering algorithms: first, it can be used as a low-cost testto The outline of the paper is as follows: in Section 2, we present determine if running a clustering algorithm is necessary or not. related work on unimodality tests. We then describe our folding Second, it can be used to improve clustering results, for example to test of unimodality in Section 3, and its experimental validation in help in parameter estimation. Section 4. Section 5 concludes the paper. 2 RELATED WORK Usage and related fields. Testing unimodality is not an end in Unimodality tests. Many statistical tests have been proposed in itself. On the contrary, it may be used as a tool for related purposes. the literature and Stoepker gave recently a nice survey of them [23]. Particularly, some clustering algorithms use such approaches to There are two kinds of tests: testing unimodality versus bimodality find relevant groups of data. In[17], the dip statistics is used to and testing unimodality versus multimodality. Once we know that find clusters in noisy dataskinny-dip ( algorithm). The authors idea a distribution is k-modal, many algorithms can be used to learn is to find all the bumps in the ecdf because they represent “high” this distribution from a few samples [3, 4]. density regions. This information comes precisely from the dip To test the “unimodality character” of a distribution, the main computation. As the dip cannot be computed in the multivariate approaches aim at estimating the distribution. They include his- case, the authors apply it on each dimension, with the issue we tograms, kernel density estimates and mixture models. raised before. Silverman’s Test is a parametric test that uses the bandwidth In [15], Kalogeratos and Likas present a combination of the dip of the kernel density estimate to test multimodality. A sufficiently statistics with the k−means algorithm (dip-means algorithm). More large bandwidth gives a smooth unimodal estimate. If we need precisely, the dip test is performed on the pairwise distances of a large amount of smoothing to find an unimodal estimate, this the points within a cluster. Indeed, a cluster is acceptable if it is indicates multimodality. This test has several drawbacks [10, 23] unimodal (unimodality assumption). If k−means provides a cluster even if we know that the input distribution is a mixture of gaussians. which is not unimodal, dip-means re-run the algorithm assuming Some corrections have been proposed in [10] and a procedure to an additional cluster (with an initial state carefully chosen). use non-gaussian kernels is given in [9]. Other approaches to estimate the k of k−means exist but they are The Excess Mass Test of Müller et al.[18] tests unimodality likely to run the clustering algorithm several times with different versus bimodality. The idea is to estimate the excess mass of the numbers of cluster so as to optimize a predefined criterion [14]. modes which can be seen as the area between the density f ¹xº and Examples of criteria are the Bayes Information Criterion (BIC), the a level L. The test compares the excess mass under unimodal and Akaike Information Criterion (AIC), the Minimum Message Length bimodal assumptions (through a statistics noted ∆). Actually, only a (MML) [7], the Minimum Description Length (MDL) [11] or the simple histogram is needed to estimate the density. Some properties Gap Statistics [24]. about the distribution of ∆ have been discovered by Cheng and We may think that these tasks are similar with testing unimodal- Hall [1] leading to a bootstrap procedure to compute p−values. ity but they are slightly different. First, clustering is stronger be- The Dip Test by Hartigan and Hartigan [13] measures the dis- cause it provides the groups of data while unimodality testing tance between the empirical cumulative distribution function (ecdf) decides whether data are gathered within a single group or not. and the set of the unimodal distributions U. The cdf of the set U are However clustering needs more information: the number of clusters and/or some parameters to define the similarity between the obser- well characterized: they are convex on an interval ¼−∞; xl ¼, then vations. Thus, unimodality testing may be viewed as a macroscopic constant on »xl ; xu ¼ and finally concave on »xu; +1». In short, the dip of an empirical cdf F corresponds roughly to the distance (with description of the data while clustering inspects it deeper. infinity norm) between F and the set U, i.e: Definition of unimodality. As presented above, unimodality is well-defined in dimension 1. In a few words, a distribution isuni- dip¹Fº = d1 ¹F; Uº = min kF − U k1 : U 2U modal iff the cdf is first convex, then flat, and finally concave.Un- fortunately this definition does not generalize in higher dimen- In [13], the uniform distribution is considered as the least fa- sions.
Load more

Recommended publications
  • Exploring Bimodality in Introductory Computer Science Performance Distributions
    EURASIA Journal of Mathematics, Science and Technology Education, 2018, 14(10), em1591 ISSN:1305-8223 (online) OPEN ACCESS Research Paper https://doi.org/10.29333/ejmste/93190 Exploring Bimodality in Introductory Computer Science Performance Distributions Ram B Basnet 1, Lori K Payne 1, Tenzin Doleck 2*, David John Lemay 2, Paul Bazelais 2 1 Colorado Mesa University, Grand Junction, Colorado, USA 2 McGill University, Montreal, Quebec, CANADA Received 7 April 2018 ▪ Revised 19 June 2018 ▪ Accepted 21 June 2018 ABSTRACT This study examines student performance distributions evidence bimodality, or whether there are two distinct populations in three introductory computer science courses grades at a four-year southwestern university in the United States for the period 2014-2017. Results suggest that computer science course grades are not bimodal. These findings counter the double hump assertion and suggest that proper course sequencing can address the needs of students with varying levels of prior knowledge and obviate the double-hump phenomenon. Studying performance helps to improve delivery of introductory computer science courses by ensuring that courses are aligned with student needs and address preconceptions and prior knowledge and experience. Keywords: computer science performance, coding, programming, double hump, grade distribution, bimodal distribution, unimodal distribution BACKGROUND In recent years, there has been a resurgence of interest in the practice of coding (Kafai & Burke, 2013), with many pushing for making it a core competency for students (Lye & Koh, 2014). There are inherent challenges in learning to code evidenced by high failure and dropout rates in programming courses (Ma, Ferguson, Roper, & Wood, 2011; (Qian & Lehman, 2017; Robins, 2010).
    [Show full text]
  • Recombining Partitions Via Unimodality Tests
    Working Paper 13-07 Departamento de Estadística Statistics and Econometrics Series (06) Universidad Carlos III de Madrid March 2013 Calle Madrid, 126 28903 Getafe (Spain) Fax (34) 91 624-98-48 RECOMBINING PARTITIONS VIA UNIMODALITY TESTS Adolfo Álvarez (1) , Daniel Peña(2) Abstract In this article we propose a recombination procedure for previously split data. It is based on the study of modes in the density of the data, since departing from unimodality can be a sign of the presence of clusters. We develop an algorithm that integrates a splitting process inherited from the SAR algorithm (Peña et al., 2004) with unimodality tests such as the dip test proposed by Hartigan and Hartigan (1985), and finally, we use a network configuration to visualize the results. We show that this can be a useful tool to detect heterogeneity in the data, but limited to univariate data because of the nature of the dip test. In a second stage we discuss the use of multivariate mode detection tests to avoid dimensionality reduction techniques such as projecting multivariate data into one dimension. The results of the application of multivariate unimodality tests show that is possible to detect the cluster structure of the data, although more research can be oriented to estimate the proper fine-tuning of some parameters of the test for a given dataset or distribution. Keywords: Cluster analysis, unimodality, dip test. (1) Álvarez, Adolfo, Departamento de Estadística, Universidad Carlos III de Madrid, C/ Madrid 126, 28903 Getafe (Madrid), Spain, e-mail: [email protected] . (2) Peña, Daniel, Departamento de Estadística, Universidad Carlos III de Madrid, C/ Madrid 126, 28903 Getafe (Madrid), Spain, e-mail: [email protected] .
    [Show full text]
  • UNIMODALITY and the DIP STATISTIC; HANDOUT I 1. Unimodality Definition. a Probability Density F Defined on the Real Line R Is C
    UNIMODALITY AND THE DIP STATISTIC; HANDOUT I 1. Unimodality Definition. A probability density f defined on the real line R is called unimodal if the following hold: (i) f is nondecreasing on the half-line ( ,b) for some real b; (ii) f is nonincreasing on the half-line (−∞a, + ) for some real a; (iii) For the largest possible b in (i), which∞ exists, and the smallest possible a in (ii), which also exists, we have a b. ≤ Here a largest possible b exists since if f is nondecreasing on ( ,bk) −∞ for all k and bk b then f is also nondecreasing on ( ,b). Also b < + since f is↑ a probability density. Likewise a smallest−∞ possible a exists∞ and is finite. In the definition of unimodal density, if a = b it is called the mode of f. It is possible that f is undefined or + there, or that f(x) + as x a and/or as x a. ∞ ↑ ∞ If a<b↑ then f is constant↓ on the interval (a,b) and we can take it to be constant on [a,b], which will be called the interval of modes of f. Any x [a,b] will be called a mode of f. When a = b the interval of modes∈ reduces to the singleton a . { } Examples. Each normal distribution N(µ,σ2) has a unimodal density with a unique mode at µ. Each uniform U[a,b] distribution has a unimodal density with an interval of modes equal to [a,b]. Consider a−1 −x the gamma density γa(x) = x e /Γ(a) for x > 0 and 0 for x 0, where the shape parameter a > 0.
    [Show full text]
  • Field Guide to Continuous Probability Distributions
    Field Guide to Continuous Probability Distributions Gavin E. Crooks v 1.0.0 2019 G. E. Crooks – Field Guide to Probability Distributions v 1.0.0 Copyright © 2010-2019 Gavin E. Crooks ISBN: 978-1-7339381-0-5 http://threeplusone.com/fieldguide Berkeley Institute for Theoretical Sciences (BITS) typeset on 2019-04-10 with XeTeX version 0.99999 fonts: Trump Mediaeval (text), Euler (math) 271828182845904 2 G. E. Crooks – Field Guide to Probability Distributions Preface: The search for GUD A common problem is that of describing the probability distribution of a single, continuous variable. A few distributions, such as the normal and exponential, were discovered in the 1800’s or earlier. But about a century ago the great statistician, Karl Pearson, realized that the known probabil- ity distributions were not sufficient to handle all of the phenomena then under investigation, and set out to create new distributions with useful properties. During the 20th century this process continued with abandon and a vast menagerie of distinct mathematical forms were discovered and invented, investigated, analyzed, rediscovered and renamed, all for the purpose of de- scribing the probability of some interesting variable. There are hundreds of named distributions and synonyms in current usage. The apparent diver- sity is unending and disorienting. Fortunately, the situation is less confused than it might at first appear. Most common, continuous, univariate, unimodal distributions can be orga- nized into a small number of distinct families, which are all special cases of a single Grand Unified Distribution. This compendium details these hun- dred or so simple distributions, their properties and their interrelations.
    [Show full text]
  • On Multivariate Unimodal Distributions
    ON MULTIVARIATE UNIMODAL DISTRIBUTIONS By Tao Dai B. Sc. (Mathematics), Wuhan University, 1982 A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE in THE FACULTY OF GRADUATE STUDIES DEPARTMENT OF STATISTICS We accept this thesis as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA June 1989 © Tao Dai, 1989 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Department of £ tasW> tj-v*> The University of British Columbia Vancouver, Canada Date ./ v^y^J. DE-6 (2/88) ABSTRACT In this thesis, Kanter's representation of multivariate unimodal distributions is shown equivalent to the usual mixture of uniform distributions on symmetric, compact and convex sets. Kanter's idea is utilized in several contexts by viewing multivariate distributions as mixtures of uniform distributions on sets of various shapes. This provides a unifying viewpoint of what is in the literature and .gives some important new classes of multivariate unimodal distributions. The closure properties of these new classes under convolution, marginality and weak convergence, etc. and their relationships with other notions of multivariate unimodality are discussed. Some interesting examples and their 2- or 3-dimensional pictures are presented.
    [Show full text]
  • Large Time Unimodality for Classical and Free Brownian Motions with Initial Distributions
    ALEA, Lat. Am. J. Probab. Math. Stat. 15, 353–374 (2018) DOI: 10.30757/ALEA.v15-15 Large time unimodality for classical and free Brownian motions with initial distributions Takahiro Hasebe and Yuki Ueda Department of Mathematics, Hokkaido University, Kita 10, Nishi 8, Kita-Ku, Sapporo, Hokkaido, 060-0810, Japan E-mail address: [email protected] URL: https://www.math.sci.hokudai.ac.jp/~thasebe/ Department of Mathematics, Hokkaido University, Kita 10, Nishi 8, Kita-Ku, Sapporo, Hokkaido, 060-0810, Japan E-mail address: [email protected] Abstract. We prove that classical and free Brownian motions with initial distri- butions are unimodal for sufficiently large time, under some assumption on the initial distributions. The assumption is almost optimal in some sense. Similar re- sults are shown for a symmetric stable process with index 1 and a positive stable process with index 1/2. We also prove that free Brownian motion with initial sym- metric unimodal distribution is unimodal, and discuss strong unimodality for free convolution. 1. Introduction A Borel measure µ on R is unimodal if there exist a R and a function f : R [0, ) which is non-decreasing on ( ,a) and non-increasing∈ on (a, ), such that→ ∞ −∞ ∞ µ(dx)= µ( a )δ + f(x) dx. (1.1) { } a The most outstanding result on unimodality is Yamazato’s theorem (Yamazato, 1978) saying that all classical selfdecomposable distributions are unimodal. After this result, in Hasebe and Thorbjørnsen (2016), Hasebe and Thorbjørnsen proved the free analog of Yamazato’s result: all freely selfdecomposable distributions are unimodal.
    [Show full text]
  • Nber Working Paper Series the Distribution of the Size
    NBER WORKING PAPER SERIES THE DISTRIBUTION OF THE SIZE OF PRICE CHANGES Alberto Cavallo Roberto Rigobon Working Paper 16760 http://www.nber.org/papers/w16760 NATIONAL BUREAU OF ECONOMIC RESEARCH 1050 Massachusetts Avenue Cambridge, MA 02138 February 2011 We are grateful to Alberto Alesina, Emmanuel Farhi, Oleksiy Kryvtsov, Greg Mankiw, Dan Sichel, and seminar participants at Harvard, MIT, NBER, AEA, BLS, ITAM, Oxford, Queen’s University at Belfast, and the Central Bank of Norway for helpful comments and suggestions. All remaining errors are ours. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research. NBER working papers are circulated for discussion and comment purposes. They have not been peer- reviewed or been subject to the review by the NBER Board of Directors that accompanies official NBER publications. © 2011 by Alberto Cavallo and Roberto Rigobon. All rights reserved. Short sections of text, not to exceed two paragraphs, may be quoted without explicit permission provided that full credit, including © notice, is given to the source. The Distribution of the Size of Price Changes Alberto Cavallo and Roberto Rigobon NBER Working Paper No. 16760 February 2011 JEL No. E00,E3 ABSTRACT Different theories of price stickiness have distinct implications on the properties of the distribution of price changes. One of those characteristics is the number of modes in the distribution. We formally test for the number of modes in the price change distribution of 32 supermarkets, spanning 23 countries and 5 continents. We present results for three modality tests: the two best-known tests in the statistical literature, Hartigan’s Dip and Silverman’s Bandwith, and a test designed in this paper, called the Proportional Mass test (PM).
    [Show full text]
  • The Basic Distributional Theory for the Product of Zero Mean Correlated
    The basic distributional theory for the product of zero mean correlated normal random variables Robert E. Gaunt∗ Abstract The product of two zero mean correlated normal random variables has received much attention in the statistics literature and appears in many application areas. However, many important distributional properties are yet to tabulated. This paper fills this gap by providing the basic distributional theory for the product of two zero mean correlated normal random variables. Keywords: Product of correlated normal random variables; distributional theory AMS 2010 Subject Classification: Primary 60E05; 62E15 1 Introduction 2 2 Let (X,Y ) be a bivariate normal random vector with zero mean vector, variances (σX , σY ) and correlation coefficient ρ. The distribution of the product Z = XY arises in nu- merous applications, with recent examples including statistics of Lagrangian power in two-dimensional turbulence (Bandi and Connaughton [3]), product confidence limits for indirect effects (MacKinnon et al. [28]) and statistical mediation analysis (MacKinnon −1 [27]). The mean Zn = n (Z1 + Z2 + + Zn), where Z1,Z2,...,Zn are independent and identical copies of Z, has also found··· application in areas such as electrical engineering (Ware and Lad [42]), astrophysics (Mangilli, Plaszczynski and Tristram [29], Watts et al. [43]) and quantum cosmology (Grishchuk [18]). The exact distribution of the product Z = XY has been studied since 1936 (Craig arXiv:2106.02897v1 [math.ST] 5 Jun 2021 [5]), with subsequent contributions coming from Aroian [1], Aroian, Taneja and Cornwell [2], Bandi and Connaughton [3], Haldane [20], Meeker et al. [31]; see Nadarajah and Pog´any [32] for an overview of these and further contributions.
    [Show full text]
  • Distributionally Robust Chance Constrained Optimal Power Flow Assuming Log-Concave Distributions
    Distributionally Robust Chance Constrained Optimal Power Flow Assuming Log-Concave Distributions B. Li 1, R. Jiang 2 and J.L. Mathieu 1 1EECS, University of Michigan 2IOE, University of Michigan November 2016 Bowen Li University of Michigan November 2016 1 / 17 Background: Chance Constrained Optimization Widely used in different problems with uncertainty But hard to accurately estimate uncertainty distributions with historical data Example: Physical constraints should be satisfied for most of the possible realizations of renewable/load uncertainty in power systems planning problems Solving methodologies Scenario approach: large number of scenarios/constraints Probabilistically robust method: overconservativeness Analytical reformulation: case dependent reliability General formulation P(L(x; ξ) ≤ 0) ≥ 1 − Bowen Li University of Michigan November 2016 2 / 17 Background: Distributionally Robust Optimization [1] Key properties A probabilistic constraint is satisfied within an ambiguity set of distributions The ambiguity set can be calibrated using the following information Moment-based information: mean, covariance, higher-order moments Density-based information: likelihood of a known probability density Distribution structure: unimodality, support, smoothness, symmetry Good trade-off between performance and computational tractability Example: distributionally robust chance constraint (DRCC) inf (L(x; ξ) ≤ 0) ≥ 1 − U P P2P U n T o P = Unimodal; EP(ξ) = µ, EP(ξξ ) = S [1] E. Delage and Y. Ye, \Distributionally Robust Optimization under Moment
    [Show full text]
  • Alpha-Skew Generalized T Distribution
    Revista Colombiana de Estadística July 2015, Volume 38, Issue 2, pp. 353 to 370 DOI: http://dx.doi.org/10.15446/rce.v38n2.51665 Alpha-Skew Generalized t Distribution Distribución t generalizada alfa sesgada Sukru Acitas1;a, Birdal Senoglu2;b, Olcay Arslan2;c 1Department of Statistics, Faculty of Science, Anadolu University, Eskisehir, Turkey 2Department of Statistics, Faculty of Science, Ankara University, Ankara, Turkey Abstract The alpha-skew normal (ASN) distribution has been proposed recently in the literature by using standard normal distribution and a skewing ap- proach. Although ASN distribution is able to model both skew and bimodal data, it is shortcoming when data has thinner or thicker tails than normal. Therefore, we propose an alpha-skew generalized t (ASGT) by using the gen- eralized t (GT) distribution and a new skewing procedure. From this point of view, ASGT can be seen as an alternative skew version of GT distribution. However, ASGT differs from the previous skew versions of GT distribution since it is able to model bimodal data sest as well as it nests most commonly used density functions. In this paper, moments and maximum likelihood estimation of the parameters of ASGT distribution are given. Skewness and kurtosis measures are derived based on the first four noncentral moments. The cumulative distribution function (cdf) of ASGT distribution is also ob- tained. In the application part of the study, two real life problems taken from the literature are modeled by using ASGT distribution. Key words: Bimodality, Kurtosis, Maximum Likelihood Estimation, Mod- eling, Skewness. Resumen La distribución normal alfa-sesgada (ASN por sus siglas en inglés) ha sido propuesta recientemente en la literatura mediante el uso de una distribución normal estándar y procedimientos de sesgo.
    [Show full text]
  • The Modes of a Mixture of Two Normal Distributions
    Silesian J. Pure Appl. Math. vol. 6, is. 1 (2016), 59–67 Grzegorz SITEK Department of Statistics, University of Economics in Katowice, Poland THE MODES OF A MIXTURE OF TWO NORMAL DISTRIBUTIONS Abstract. Mixture distributions arise naturally where a statistical pop- ulation contains two or more subpopulations. Finite mixture distributions refer to composite distributions constructed by mixing a number K of com- ponent distributions. The first account of mixture data being analyzed was documented by Pearson in 1894. We consider the distribution of a mix- ture of two normal distributions and investigate the conditions for which the distribution is bimodal. This paper presents a procedure for answer- ing the question of whether a mixture of two normal distributions which five known parameters µ1, µ2, σ1, σ2, p is unimodal or not. For finding the modes, a simple iterative procedure is given. This article presents the possibility of estimation of modes using biaverage. 1. Introduction Mixture models have been widely used in econometrics and social science, and the theories for mixture models have been well studied Lindsay (see [5]). The importance of the research for unimodality or bimodality in statistics have been described by Murphy (see [6]). Consider f(x, p)= pf (x)+(1 p)f (x) (1) 1 − 2 2010 Mathematics Subject Classification: 60E05, 62E99. Keywords: mixture of normal distributions, bimodal distributions, biaverage. Corresponding author: G. Sitek ([email protected]). Received: 25.07.2016. 60 G. Sitek where, for i =1, 2, 1 − − 2 2 (x µi) /2σi fi(x)= e σi√2π with 0 <p< 1. The function f(x, p) is the probability density function of a mixture of two normal distributions.
    [Show full text]
  • Distributionally Robust Chance Constrained Optimal Power
    1 Distributionally Robust Chance Constrained Optimal Power Flow Assuming Unimodal Distributions with Misspecified Modes Bowen Li, Student Member, IEEE, Ruiwei Jiang, Member, IEEE, and Johanna L. Mathieu, Member, IEEE Abstract—Chance constrained optimal power flow (CC-OPF) approximation [8], [9], which provides a priori guarantees formulations have been proposed to minimize operational costs on reliability, require the constraints to be satisfied over a while controlling the risk arising from uncertainties like renew- large number of uncertainty samples. The solutions from these able generation and load consumption. To solve CC-OPF, we often need access to the (true) joint probability distribution of approaches are usually overly conservative with high costs all uncertainties, which is rarely known in practice. A solution [7], [10]. Another popular approach is to assume that the based on a biased estimate of the distribution can result in uncertainties follow a parametric distribution such as Gaussian poor reliability. To overcome this challenge, recent work has [4], [5], [7]. The resulting CC-OPF is often easier to solve explored distributionally robust chance constraints, in which the but the solution may have low reliability unless the assumed chance constraints are satisfied over a family of distributions called the ambiguity set. Commonly, ambiguity sets are only probability distribution happens to be close to the true one. based on moment information (e.g., mean and covariance) of the As an alternative, distributionally robust chance constrained random variables; however, specifying additional characteristics (DRCC) OPF models do not depend on a single estimate of the random variables reduces conservatism and cost. Here, we of the probability distribution [10]–[17].
    [Show full text]