Simple New Proofs of the Characteristic Functions of the F and Skew-Normal Distributions
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Connection Between Dirichlet Distributions and a Scale-Invariant Probabilistic Model Based on Leibniz-Like Pyramids
Home Search Collections Journals About Contact us My IOPscience Connection between Dirichlet distributions and a scale-invariant probabilistic model based on Leibniz-like pyramids This content has been downloaded from IOPscience. Please scroll down to see the full text. View the table of contents for this issue, or go to the journal homepage for more Download details: IP Address: 152.84.125.254 This content was downloaded on 22/12/2014 at 23:17 Please note that terms and conditions apply. Journal of Statistical Mechanics: Theory and Experiment Connection between Dirichlet distributions and a scale-invariant probabilistic model based on J. Stat. Mech. Leibniz-like pyramids A Rodr´ıguez1 and C Tsallis2,3 1 Departamento de Matem´atica Aplicada a la Ingenier´ıa Aeroespacial, Universidad Polit´ecnica de Madrid, Pza. Cardenal Cisneros s/n, 28040 Madrid, Spain 2 Centro Brasileiro de Pesquisas F´ısicas and Instituto Nacional de Ciˆencia e Tecnologia de Sistemas Complexos, Rua Xavier Sigaud 150, 22290-180 Rio de Janeiro, Brazil (2014) P12027 3 Santa Fe Institute, 1399 Hyde Park Road, Santa Fe, NM 87501, USA E-mail: [email protected] and [email protected] Received 4 November 2014 Accepted for publication 30 November 2014 Published 22 December 2014 Online at stacks.iop.org/JSTAT/2014/P12027 doi:10.1088/1742-5468/2014/12/P12027 Abstract. We show that the N limiting probability distributions of a recently introduced family of d→∞-dimensional scale-invariant probabilistic models based on Leibniz-like (d + 1)-dimensional hyperpyramids (Rodr´ıguez and Tsallis 2012 J. Math. Phys. 53 023302) are given by Dirichlet distributions for d = 1, 2, ... -
Distribution of Mutual Information
Distribution of Mutual Information Marcus Hutter IDSIA, Galleria 2, CH-6928 Manno-Lugano, Switzerland [email protected] http://www.idsia.ch/-marcus Abstract The mutual information of two random variables z and J with joint probabilities {7rij} is commonly used in learning Bayesian nets as well as in many other fields. The chances 7rij are usually estimated by the empirical sampling frequency nij In leading to a point es timate J(nij In) for the mutual information. To answer questions like "is J (nij In) consistent with zero?" or "what is the probability that the true mutual information is much larger than the point es timate?" one has to go beyond the point estimate. In the Bayesian framework one can answer these questions by utilizing a (second order) prior distribution p( 7r) comprising prior information about 7r. From the prior p(7r) one can compute the posterior p(7rln), from which the distribution p(Iln) of the mutual information can be cal culated. We derive reliable and quickly computable approximations for p(Iln). We concentrate on the mean, variance, skewness, and kurtosis, and non-informative priors. For the mean we also give an exact expression. Numerical issues and the range of validity are discussed. 1 Introduction The mutual information J (also called cross entropy) is a widely used information theoretic measure for the stochastic dependency of random variables [CT91, SooOO] . It is used, for instance, in learning Bayesian nets [Bun96, Hec98] , where stochasti cally dependent nodes shall be connected. The mutual information defined in (1) can be computed if the joint probabilities {7rij} of the two random variables z and J are known. -
Lecture 24: Dirichlet Distribution and Dirichlet Process
Stat260: Bayesian Modeling and Inference Lecture Date: April 28, 2010 Lecture 24: Dirichlet distribution and Dirichlet Process Lecturer: Michael I. Jordan Scribe: Vivek Ramamurthy 1 Introduction In the last couple of lectures, in our study of Bayesian nonparametric approaches, we considered the Chinese Restaurant Process, Bayesian mixture models, stick breaking, and the Dirichlet process. Today, we will try to gain some insight into the connection between the Dirichlet process and the Dirichlet distribution. 2 The Dirichlet distribution and P´olya urn First, we note an important relation between the Dirichlet distribution and the Gamma distribution, which is used to generate random vectors which are Dirichlet distributed. If, for i ∈ {1, 2, · · · , K}, Zi ∼ Gamma(αi, β) independently, then K K S = Zi ∼ Gamma αi, β i=1 i=1 ! X X and V = (V1, · · · ,VK ) = (Z1/S, · · · ,ZK /S) ∼ Dir(α1, · · · , αK ) Now, consider the following P´olya urn model. Suppose that • Xi - color of the ith draw • X - space of colors (discrete) • α(k) - number of balls of color k initially in urn. We then have that α(k) + j<i δXj (k) p(Xi = k|X1, · · · , Xi−1) = α(XP) + i − 1 where δXj (k) = 1 if Xj = k and 0 otherwise, and α(X ) = k α(k). It may then be shown that P n α(x1) α(xi) + j<i δXj (xi) p(X1 = x1, X2 = x2, · · · , X = x ) = n n α(X ) α(X ) + i − 1 i=2 P Y α(1)[α(1) + 1] · · · [α(1) + m1 − 1]α(2)[α(2) + 1] · · · [α(2) + m2 − 1] · · · α(C)[α(C) + 1] · · · [α(C) + m − 1] = C α(X )[α(X ) + 1] · · · [α(X ) + n − 1] n where 1, 2, · · · , C are the distinct colors that appear in x1, · · · ,xn and mk = i=1 1{Xi = k}. -
The Exciting Guide to Probability Distributions – Part 2
The Exciting Guide To Probability Distributions – Part 2 Jamie Frost – v1.1 Contents Part 2 A revisit of the multinomial distribution The Dirichlet Distribution The Beta Distribution Conjugate Priors The Gamma Distribution We saw in the last part that the multinomial distribution was over counts of outcomes, given the probability of each outcome and the total number of outcomes. xi f(x1, ... , xk | n, p1, ... , pk)= [n! / ∏xi!] ∏pi The count of The probability of each outcome. each outcome. That’s all smashing, but suppose we wanted to know the reverse, i.e. the probability that the distribution underlying our random variable has outcome probabilities of p1, ... , pk, given that we observed each outcome x1, ... , xk times. In other words, we are considering all the possible probability distributions (p1, ... , pk) that could have generated these counts, rather than all the possible counts given a fixed distribution. Initial attempt at a probability mass function: Just swap the domain and the parameters: The RHS is exactly the same. xi f(p1, ... , pk | n, x1, ... , xk )= [n! / ∏xi!] ∏pi Notational convention is that we define the support as a vector x, so let’s relabel p as x, and the counts x as α... αi f(x1, ... , xk | n, α1, ... , αk )= [n! / ∏ αi!] ∏xi We can define n just as the sum of the counts: αi f(x1, ... , xk | α1, ... , αk )= [(∑αi)! / ∏ αi!] ∏xi But wait, we’re not quite there yet. We know that probabilities have to sum to 1, so we need to restrict the domain we can draw from: s.t. -
Uniqueness of the Gaussian Orthogonal Ensemble
Uniqueness of the Gaussian Orthogonal Ensemble ´ Jos´e Angel S´anchez G´omez∗, Victor Amaya Carvajal†. December, 2017. A known result in random matrix theory states the following: Given a ran- dom Wigner matrix X which belongs to the Gaussian Orthogonal Ensemble (GOE), then such matrix X has an invariant distribution under orthogonal conjugations. The goal of this work is to prove the converse, that is, if X is a symmetric random matrix such that it is invariant under orthogonal conjuga- tions, then such matrix X belongs to the GOE. We will prove this using some elementary properties of the characteristic function of random variables. 1 Introduction We will prove one of the main characterization of the matrices that belong to the Gaussian Orthogonal Ensemble. This work was done as a final project for the Random Matrices graduate course at the Centro de Investigaci´on en Matem´aticas, A.C. (CIMAT), M´exico. 2 Background I We will denote by n m(F), the set of matrices of size n m over the field F. The matrix M × × Id d will denote the identity matrix of the set d d(F). × M × Definition 1. Denote by (n) the set of orthogonal matrices of n n(R). This means, ⊺ O ⊺ M × if O (n) then O O = OO = In n. ∈ O × ⊺ arXiv:1901.09257v3 [math.PR] 1 Oct 2019 Definition 2. A symmetric matrix X is a square matrix such that X = X. We will denote the set of symmetric n n matrices by Sn. × Definition 3. [Gaussian Orthogonal Ensemble (GOE)]. -
Stat 5101 Lecture Slides: Deck 8 Dirichlet Distribution
Stat 5101 Lecture Slides: Deck 8 Dirichlet Distribution Charles J. Geyer School of Statistics University of Minnesota This work is licensed under a Creative Commons Attribution- ShareAlike 4.0 International License (http://creativecommons.org/ licenses/by-sa/4.0/). 1 The Dirichlet Distribution The Dirichlet Distribution is to the beta distribution as the multi- nomial distribution is to the binomial distribution. We get it by the same process that we got to the beta distribu- tion (slides 128{137, deck 3), only multivariate. Recall the basic theorem about gamma and beta (same slides referenced above). 2 The Dirichlet Distribution (cont.) Theorem 1. Suppose X and Y are independent gamma random variables X ∼ Gam(α1; λ) Y ∼ Gam(α2; λ) then U = X + Y V = X=(X + Y ) are independent random variables and U ∼ Gam(α1 + α2; λ) V ∼ Beta(α1; α2) 3 The Dirichlet Distribution (cont.) Corollary 1. Suppose X1;X2;:::; are are independent gamma random variables with the same shape parameters Xi ∼ Gam(αi; λ) then the following random variables X1 ∼ Beta(α1; α2) X1 + X2 X1 + X2 ∼ Beta(α1 + α2; α3) X1 + X2 + X3 . X1 + ··· + Xd−1 ∼ Beta(α1 + ··· + αd−1; αd) X1 + ··· + Xd are independent and have the asserted distributions. 4 The Dirichlet Distribution (cont.) From the first assertion of the theorem we know X1 + ··· + Xk−1 ∼ Gam(α1 + ··· + αk−1; λ) and is independent of Xk. Thus the second assertion of the theorem says X1 + ··· + Xk−1 ∼ Beta(α1 + ··· + αk−1; αk)(∗) X1 + ··· + Xk and (∗) is independent of X1 + ··· + Xk. That proves the corollary. 5 The Dirichlet Distribution (cont.) Theorem 2. -
Parameter Specification of the Beta Distribution and Its Dirichlet
%HWD'LVWULEXWLRQVDQG,WV$SSOLFDWLRQV 3DUDPHWHU6SHFLILFDWLRQRIWKH%HWD 'LVWULEXWLRQDQGLWV'LULFKOHW([WHQVLRQV 8WLOL]LQJ4XDQWLOHV -5HQpYDQ'RUSDQG7KRPDV$0D]]XFKL (Submitted January 2003, Revised March 2003) I. INTRODUCTION.................................................................................................... 1 II. SPECIFICATION OF PRIOR BETA PARAMETERS..............................................5 A. Basic Properties of the Beta Distribution...............................................................6 B. Solving for the Beta Prior Parameters...................................................................8 C. Design of a Numerical Procedure........................................................................12 III. SPECIFICATION OF PRIOR DIRICHLET PARAMETERS................................. 17 A. Basic Properties of the Dirichlet Distribution...................................................... 18 B. Solving for the Dirichlet prior parameters...........................................................20 IV. SPECIFICATION OF ORDERED DIRICHLET PARAMETERS...........................22 A. Properties of the Ordered Dirichlet Distribution................................................. 23 B. Solving for the Ordered Dirichlet Prior Parameters............................................ 25 C. Transforming the Ordered Dirichlet Distribution and Numerical Stability ......... 27 V. CONCLUSIONS........................................................................................................ 31 APPENDIX................................................................................................................... -
Generalized Dirichlet Distributions on the Ball and Moments
Generalized Dirichlet distributions on the ball and moments F. Barthe,∗ F. Gamboa,† L. Lozada-Chang‡and A. Rouault§ September 3, 2018 Abstract The geometry of unit N-dimensional ℓp balls (denoted here by BN,p) has been intensively investigated in the past decades. A particular topic of interest has been the study of the asymptotics of their projections. Apart from their intrinsic interest, such questions have applications in several probabilistic and geometric contexts [BGMN05]. In this paper, our aim is to revisit some known results of this flavour with a new point of view. Roughly speaking, we will endow BN,p with some kind of Dirichlet distribution that generalizes the uniform one and will follow the method developed in [Ski67], [CKS93] in the context of the randomized moment space. The main idea is to build a suitable coordinate change involving independent random variables. Moreover, we will shed light on connections between the randomized balls and the randomized moment space. n Keywords: Moment spaces, ℓp -balls, canonical moments, Dirichlet dis- tributions, uniform sampling. AMS classification: 30E05, 52A20, 60D05, 62H10, 60F10. arXiv:1002.1544v2 [math.PR] 21 Oct 2010 1 Introduction The starting point of our work is the study of the asymptotic behaviour of the moment spaces: 1 [0,1] j MN = t µ(dt) : µ ∈M1([0, 1]) , (1) (Z0 1≤j≤N ) ∗Equipe de Statistique et Probabilit´es, Institut de Math´ematiques de Toulouse (IMT) CNRS UMR 5219, Universit´ePaul-Sabatier, 31062 Toulouse cedex 9, France. e-mail: [email protected] †IMT e-mail: [email protected] ‡Facultad de Matem´atica y Computaci´on, Universidad de la Habana, San L´azaro y L, Vedado 10400 C.Habana, Cuba. -
Chap 3 : Two Random Variables
Chap 3: Two Random Variables Chap 3 : Two Random Variables Chap 3.1: Distribution Functions of Two RVs In many experiments, the observations are expressible not as a single quantity, but as a family of quantities. For example to record the height and weight of each person in a community or the number of people and the total income in a family, we need two numbers. Let X and Y denote two random variables based on a probability model .(Ω,F,P). Then x2 P (x1 <X(ξ) ≤ x2)=FX (x2) − FX (x1)= fX (x) dx x1 and y2 P (y1 <Y(ξ) ≤ y2)=FY (y2) − FY (y1)= fY (y) dy y1 What about the probability that the pair of RVs (X, Y ) belongs to an arbitrary region D?In other words, how does one estimate, for example P [(x1 <X(ξ) ≤ x2) ∩ (y1 <Y(ξ) ≤ y2)] =? Towards this, we define the joint probability distribution function of X and Y to be FXY (x, y)=P (X ≤ x, Y ≤ y) ≥ 0 (1) 1 Chap 3: Two Random Variables where x and y are arbitrary real numbers. Properties 1. FXY (−∞,y)=FXY (x, −∞)=0,FXY (+∞, +∞)=1 (2) since (X(ξ) ≤−∞,Y(ξ) ≤ y) ⊂ (X(ξ) ≤−∞), we get FXY (−∞,y) ≤ P (X(ξ) ≤−∞)=0 Similarly, (X(ξ) ≤ +∞,Y(ξ) ≤ +∞)=Ω,wegetFXY (+∞, +∞)=P (Ω) = 1. 2. P (x1 <X(ξ) ≤ x2,Y(ξ) ≤ y)=FXY (x2,y) − FXY (x1,y) (3) P (X(ξ) ≤ x, y1 <Y(ξ) ≤ y2)=FXY (x, y2) − FXY (x, y1) (4) To prove (3), we note that for x2 >x1 (X(ξ) ≤ x2,Y(ξ) ≤ y)=(X(ξ) ≤ x1,Y(ξ) ≤ y) ∪ (x1 <X(ξ) ≤ x2,Y(ξ) ≤ y) 2 Chap 3: Two Random Variables and the mutually exclusive property of the events on the right side gives P (X(ξ) ≤ x2,Y(ξ) ≤ y)=P (X(ξ) ≤ x1,Y(ξ) ≤ y)+P (x1 <X(ξ) ≤ x2,Y(ξ) ≤ y) which proves (3). -
A Brief on Characteristic Functions
Missouri University of Science and Technology Scholars' Mine Graduate Student Research & Creative Works Student Research & Creative Works 01 Dec 2020 A Brief on Characteristic Functions Austin G. Vandegriffe Follow this and additional works at: https://scholarsmine.mst.edu/gradstudent_works Part of the Applied Mathematics Commons, and the Probability Commons Recommended Citation Vandegriffe, Austin G., "A Brief on Characteristic Functions" (2020). Graduate Student Research & Creative Works. 2. https://scholarsmine.mst.edu/gradstudent_works/2 This Presentation is brought to you for free and open access by Scholars' Mine. It has been accepted for inclusion in Graduate Student Research & Creative Works by an authorized administrator of Scholars' Mine. This work is protected by U. S. Copyright Law. Unauthorized use including reproduction for redistribution requires the permission of the copyright holder. For more information, please contact [email protected]. A Brief on Characteristic Functions A Presentation for Harmonic Analysis Missouri S&T : Rolla, MO Presentation by Austin G. Vandegriffe 2020 Contents 1 Basic Properites of Characteristic Functions 1 2 Inversion Formula 5 3 Convergence & Continuity of Characteristic Functions 9 4 Convolution of Measures 13 Appendix A Topology 17 B Measure Theory 17 B.1 Basic Measure Theory . 17 B.2 Convergence in Measure & Its Consequences . 20 B.3 Derivatives of Measures . 22 C Analysis 25 i Notation 8 For all 8P For P-almost all, where P is a measure 9 There exists () If and only if U Disjoint union -
Spatially Constrained Student's T-Distribution Based Mixture Model
J Math Imaging Vis (2018) 60:355–381 https://doi.org/10.1007/s10851-017-0759-8 Spatially Constrained Student’s t-Distribution Based Mixture Model for Robust Image Segmentation Abhirup Banerjee1 · Pradipta Maji1 Received: 8 January 2016 / Accepted: 5 September 2017 / Published online: 21 September 2017 © Springer Science+Business Media, LLC 2017 Abstract The finite Gaussian mixture model is one of the Keywords Segmentation · Student’s t-distribution · most popular frameworks to model classes for probabilistic Expectation–maximization · Hidden Markov random field model-based image segmentation. However, the tails of the Gaussian distribution are often shorter than that required to model an image class. Also, the estimates of the class param- 1 Introduction eters in this model are affected by the pixels that are atypical of the components of the fitted Gaussian mixture model. In In image processing, segmentation refers to the process this regard, the paper presents a novel way to model the image of partitioning an image space into some non-overlapping as a mixture of finite number of Student’s t-distributions for meaningful homogeneous regions. It is an indispensable step image segmentation problem. The Student’s t-distribution for many image processing problems, particularly for med- provides a longer tailed alternative to the Gaussian distri- ical images. Segmentation of brain images into three main bution and gives reduced weight to the outlier observations tissue classes, namely white matter (WM), gray matter (GM), during the parameter estimation step in finite mixture model. and cerebro-spinal fluid (CSF), is important for many diag- Incorporating the merits of Student’s t-distribution into the nostic studies. -
Random Variables and Probability Distributions
Schaum's Outline of Probability and Statistics CHAPTERCHAPTER 122 Random Variables and Probability Distributions Random Variables Suppose that to each point of a sample space we assign a number. We then have a function defined on the sam- ple space. This function is called a random variable (or stochastic variable) or more precisely a random func- tion (stochastic function). It is usually denoted by a capital letter such as X or Y. In general, a random variable has some specified physical, geometrical, or other significance. EXAMPLE 2.1 Suppose that a coin is tossed twice so that the sample space is S ϭ {HH, HT, TH, TT}. Let X represent the number of heads that can come up. With each sample point we can associate a number for X as shown in Table 2-1. Thus, for example, in the case of HH (i.e., 2 heads), X ϭ 2 while for TH (1 head), X ϭ 1. It follows that X is a random variable. Table 2-1 Sample Point HH HT TH TT X 2110 It should be noted that many other random variables could also be defined on this sample space, for example, the square of the number of heads or the number of heads minus the number of tails. A random variable that takes on a finite or countably infinite number of values (see page 4) is called a dis- crete random variable while one which takes on a noncountably infinite number of values is called a nondiscrete random variable. Discrete Probability Distributions Let X be a discrete random variable, and suppose that the possible values that it can assume are given by x1, x2, x3, .