DOCSLIB.ORG

Beta-Negative Binomial Process and Poisson Factor Analysis

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Beta-Negative Binomial Process and Poisson Factor Analysis Beta-Negative Binomial Process and Poisson Factor Analysis Mingyuan Zhou Lauren A. Hannah† David B. Dunson† Lawrence Carin Department of ECE, †Department of Statistical Science, Duke University, Durham NC 27708, USA Abstract to be prohibitive in high-dimensional settings and the Gaussian assumption is restrictive for count data that are discrete and nonnegative, have limited ranges, and A beta-negative binomial (BNB) process is often present overdispersion. In this paper we pro- proposed, leading to a beta-gamma-Poisson pose a flexible new nonparametric Bayesian prior to process, which may be viewed as a “multi- address these problems, the beta-negative binomial scoop” generalization of the beta-Bernoulli (BNB) process. process. The BNB process is augmented into a beta-gamma-gamma-Poisson hierar- Using completely random measures (Kingman, 1967), chical structure, and applied as a nonpara- Thibaux and Jordan (2007) generalize the beta process metric Bayesian prior for an infinite Poisson defined on [0, 1] R+ by Hjort (1990) to a general prod- factor analysis model. A finite approximation uct space [0, 1] ×Ω, and define a Bernoulli process on an for the beta process L´evyrandom measure is atomic beta process× hazard measure to model binary constructed for convenient implementation. outcomes. They further show that the beta-Bernoulli Efficient MCMC computations are performed process is the underlying de Finetti mixing distribu- with data augmentation and marginalization tion for the Indian buffet process (IBP) of Griffiths techniques. Encouraging results are shown and Ghahramani (2005). To model count variables, on document count matrix factorization. we extend the measure space of the beta process to [0, 1] R+ Ω and introduce a negative binomial pro- cess,× leading× to the BNB process. We show that the 1 Introduction BNB process can be augmented into a beta-gamma- Poisson process, and that this process may be inter- Count data appear in many settings. Problems in- preted in terms of a “multi-scoop” IBP. Specifically, clude predicting future demand for medical care based each “customer” visits an infinite set of dishes on a on past use (Cameron et al., 1988; Deb and Trivedi, buffet line, and rather than simply choosing to select 1997), species sampling (National Audubon Society) certain dishes off the buffet (as in the IBP), the cus- and topic modeling of document corpora (Blei et al., tomer may select multiple scoops of each dish, with 2003). Poisson and negative binomial distributions the number of scoops controlled by a negative bino- are typical choices for univariate and repeated mea- mial distribution with dish-dependent hyperparame- sures count data; however, multivariate extensions in- ters. As discussed below, the use of a negative bino- corporating latent variables (latent counts) are under mial distribution for modeling the number of scoops developed. Latent variable models under the Gaus- is more general than using a Poisson distribution, as sian assumption, such as principal component anal- one may control both the mean and variance of the ysis and factor analysis, are widely used to discover counts, and allow overdispersion. This representation low-dimensional data structure (Lawrence, 2005; Tip- is particularly useful for discrete latent variable mod- ping and Bishop, 1999; West, 2003; Zhou et al., 2009). els, where each latent feature is not simply present or There has been some work on exponential family la- absent, but contributes a distinct count to each obser- tent factor models that incorporate Gaussian latent vation. variables (Dunson, 2000, 2003; Moustaki and Knott, We use the BNB process to construct an infinite dis- 2000; Sammel et al., 1997), but computation tends crete latent variable model called Poisson factor anal- ysis (PFA), where an observed count is linked to its Appearing in Proceedings of the 15th International Con- latent parameters with a Poisson distribution. To en- ference on Artificial Intelligence and Statistics (AISTATS) hance model flexibility, we place a gamma prior on 2012, La Palma, Canary Islands. Volume XX of JMLR: W&CP XX. Copyright 2012 by the authors. the Poisson rate parameter, leading to a negative bi- 1462 Beta-Negative Binomial Process and Poisson Factor Analysis nomial distribution. The BNB process is formulated in with ν(dpdω) ν+π(dpdω). A random signed mea- a beta-gamma-gamma-Poisson hierarchical structure, sure satisfying≡ (2) is called a L´evyrandom measure. with which we construct an infinite PFA model for MoreL generally, if the L´evymeasure ν(dpdω) satisfies count matrix factorization. We test PFA with various priors for document count matrix factorization, mak- (1 p )ν(dpdω) < (3) S ∧ | | ∞ ing connections to previous models; here a latent count ZZR× for each compact S Ω, it need not be finite for the assigned to a factor (topic) is the number of times that L´evyrandom measure⊂ to be well defined; the nota- factor appears in the document. tion 1 p denotes minL 1, p . A nonnegative L´evy ∧ | | { | |} The contributions of this paper are: 1) an extension random measure satisfying (3) was called a com- L of the beta process to a marked space, to produce the pletely random measure (CRM) by Kingman (1967, beta-negative binomial (BNB) process; 2) efficient in- 1993) and an additive random measure by C¸inlar ference for the BNB process; and 3) a flexible model (2011). It was introduced for machine learning by for count matrix factorization, which accurately cap- Thibaux and Jordan (2007) and Jordan (2010). tures topics with diverse characteristics when applied to topic modeling of document corpora. 2.3 Beta Process The beta process (BP) was defined by Hjort (1990) for 2 Preliminaries survival analysis with Ω = R+. Thibaux and Jordan (2007) generalized the process to an arbitrary measur- 2.1 Negative Binomial Distribution able space Ω by defining a CRM B on a product space [0, 1] Ω with the L´evymeasure The Poisson distribution X Pois(λ) is commonly × ∼ 1 c 1 used for modeling count data. It has the probability ν (dpdω) = cp− (1 p) − dpB (dω). (4) BP − 0 mass function f (k) = eλλk/k!, where k 0, 1,... , X ∈ { } Here c > 0 is a concentration parameter (or concentra- with both the mean and variance equal to λ. A gamma tion function if c is a function of ω), B0 is a continuous distribution with shape r and scale p/(1 p) can be − finite measure over Ω, called the base measure, and placed as a prior on λ to produce a negative binomial α = B0(Ω) is the mass parameter. Since νBP(dpdω) (a.k.a, gamma-Poisson) distribution as integrates to infinity but satisfies (3), a countably in- finite number of i.i.d. random points (pk, ωk) k=1, ∞ { } ∞ fX (k) = Pois(k; λ)Gamma (λ; r, p/(1 p)) dλ are obtained from the Poisson process with mean mea- 0 − Z sure νBP and ∞ pk is finite, where the atom ωk Ω Γ(r + k) k=1 ∈ = (1 p)rpk (1) and its weight pk [0, 1]. Therefore, we can express a P ∈ k!Γ(r) − ∞ BP draw, B BP(c, B0), as B = k=1 pkδωk , where δ is a unit∼ measure at the atom ω . If B is dis- where Γ( ) denotes the gamma function. Parame- ωk k 0 · crete (atomic) and of the form B P= q δ , then terized by r > 0 and p (0, 1), this distribution 0 k k ωk ∈ 2 B = p δ with p Beta(cq , c(1 q )). If B X NB(r, p) has a variance rp/(1 p) larger than k k ωk k k k 0 ∼ − is mixed discrete-continuous,∼ B is theP sum− of the two the mean rp/(1 p), and thus it is usually favored P for modeling overdispersed− count data. More detailed independent contributions. discussions about the negative binomial and related distributions and the corresponding stochastic pro- 3 The Beta Process and the Negative cesses defined on R+ can be found in Kozubowski and Binomial Process Podg´orski(2009) and Barndorff-Nielsen et al. (2010). Let B be a BP draw as defined in Sec. 2.3, and there- ∞ 2.2 L´evyRandom Measures fore B = k=1 pkδωk . A Bernoulli process BeP(B) has atoms appearing at the same locations as those of Beta-negative binomial processes are created using P B; it assigns atom ωk unit mass with probability pk, L´evy random measures. Following Wolpert et al. and zero mass with probability 1 pk, i.e., Bernoulli. + (2011), for any ν 0 and any probability distri- Consequently, each draw from BeP(− B) selects a (fi- ≥ + bution π(dpdω) on R Ω, let K Pois(ν ) and nite) subset of the atoms in B. This construction is iid × ∼ (pk, ωk) 1 k K π(dpdω). Defining 1A(ωk) as be- attractive because the beta distribution is a conjugate { } ≤ ≤ ∼ ing one if ωk A and zero otherwise, the random prior for the Bernoulli distribution. It has diverse ap- measure (A) ∈ K 1 (ω )p assigns independent plications including document classification (Thibaux L ≡ k=1 A k k infinitely divisible random variables (Ai) to disjoint and Jordan, 2007), dictionary learning (Zhou et al., Borel sets A Ω,P with characteristicL functions 2012, 2009, 2011) and topic modeling (Li et al., 2011). i ⊂ it (A) itp The beta distribution is also the conjugate prior for E e L = exp (e 1)ν(dpdω) (2) A − the negative binomial distribution parameter p, which ZZR× 1463 Mingyuan Zhou, Lauren A. Hannah, David B. Dunson, Lawrence Carin suggests coupling the beta process with the negative 3.2 Model Properties binomial process. Further, for modeling flexibility it is Assume we already observe X . Since the beta also desirable to place a prior on the negative-binomial { i}i=1,n parameter r, this motivating a marked beta process.
Load more

Recommended publications
  • The Exponential Family 1 Definition
    The Exponential Family David M. Blei Columbia University November 9, 2016 The exponential family is a class of densities (Brown, 1986). It encompasses many familiar forms of likelihoods, such as the Gaussian, Poisson, multinomial, and Bernoulli. It also encompasses their conjugate priors, such as the Gamma, Dirichlet, and beta. 1 Definition A probability density in the exponential family has this form p.x / h.x/ exp >t.x/ a./ ; (1) j D f g where is the natural parameter; t.x/ are sufficient statistics; h.x/ is the “base measure;” a./ is the log normalizer. Examples of exponential family distributions include Gaussian, gamma, Poisson, Bernoulli, multinomial, Markov models. Examples of distributions that are not in this family include student-t, mixtures, and hidden Markov models. (We are considering these families as distributions of data. The latent variables are implicitly marginalized out.) The statistic t.x/ is called sufficient because the probability as a function of only depends on x through t.x/. The exponential family has fundamental connections to the world of graphical models (Wainwright and Jordan, 2008). For our purposes, we’ll use exponential 1 families as components in directed graphical models, e.g., in the mixtures of Gaussians. The log normalizer ensures that the density integrates to 1, Z a./ log h.x/ exp >t.x/ d.x/ (2) D f g This is the negative logarithm of the normalizing constant. The function h.x/ can be a source of confusion. One way to interpret h.x/ is the (unnormalized) distribution of x when 0. It might involve statistics of x that D are not in t.x/, i.e., that do not vary with the natural parameter.
    [Show full text]
  • A Skew Extension of the T-Distribution, with Applications
    J. R. Statist. Soc. B (2003) 65, Part 1, pp. 159–174 A skew extension of the t-distribution, with applications M. C. Jones The Open University, Milton Keynes, UK and M. J. Faddy University of Birmingham, UK [Received March 2000. Final revision July 2002] Summary. A tractable skew t-distribution on the real line is proposed.This includes as a special case the symmetric t-distribution, and otherwise provides skew extensions thereof.The distribu- tion is potentially useful both for modelling data and in robustness studies. Properties of the new distribution are presented. Likelihood inference for the parameters of this skew t-distribution is developed. Application is made to two data modelling examples. Keywords: Beta distribution; Likelihood inference; Robustness; Skewness; Student’s t-distribution 1. Introduction Student’s t-distribution occurs frequently in statistics. Its usual derivation and use is as the sam- pling distribution of certain test statistics under normality, but increasingly the t-distribution is being used in both frequentist and Bayesian statistics as a heavy-tailed alternative to the nor- mal distribution when robustness to possible outliers is a concern. See Lange et al. (1989) and Gelman et al. (1995) and references therein. It will often be useful to consider a further alternative to the normal or t-distribution which is both heavy tailed and skew. To this end, we propose a family of distributions which includes the symmetric t-distributions as special cases, and also includes extensions of the t-distribution, still taking values on the whole real line, with non-zero skewness. Let a>0 and b>0be parameters.
    [Show full text]
  • On a Problem Connected with Beta and Gamma Distributions by R
    ON A PROBLEM CONNECTED WITH BETA AND GAMMA DISTRIBUTIONS BY R. G. LAHA(i) 1. Introduction. The random variable X is said to have a Gamma distribution G(x;0,a)if du for x > 0, (1.1) P(X = x) = G(x;0,a) = JoT(a)" 0 for x ^ 0, where 0 > 0, a > 0. Let X and Y be two independently and identically distributed random variables each having a Gamma distribution of the form (1.1). Then it is well known [1, pp. 243-244], that the random variable W = X¡iX + Y) has a Beta distribution Biw ; a, a) given by 0 for w = 0, (1.2) PiW^w) = Biw;x,x)=\ ) u"-1il-u)'-1du for0<w<l, Ío T(a)r(a) 1 for w > 1. Now we can state the converse problem as follows : Let X and Y be two independently and identically distributed random variables having a common distribution function Fix). Suppose that W = Xj{X + Y) has a Beta distribution of the form (1.2). Then the question is whether £(x) is necessarily a Gamma distribution of the form (1.1). This problem was posed by Mauldon in [9]. He also showed that the converse problem is not true in general and constructed an example of a non-Gamma distribution with this property using the solution of an integral equation which was studied by Goodspeed in [2]. In the present paper we carry out a systematic investigation of this problem. In §2, we derive some general properties possessed by this class of distribution laws Fix).
    [Show full text]
  • A Family of Skew-Normal Distributions for Modeling Proportions and Rates with Zeros/Ones Excess
    S S symmetry Article A Family of Skew-Normal Distributions for Modeling Proportions and Rates with Zeros/Ones Excess Guillermo Martínez-Flórez 1, Víctor Leiva 2,* , Emilio Gómez-Déniz 3 and Carolina Marchant 4 1 Departamento de Matemáticas y Estadística, Facultad de Ciencias Básicas, Universidad de Córdoba, Montería 14014, Colombia; [email protected] 2 Escuela de Ingeniería Industrial, Pontificia Universidad Católica de Valparaíso, 2362807 Valparaíso, Chile 3 Facultad de Economía, Empresa y Turismo, Universidad de Las Palmas de Gran Canaria and TIDES Institute, 35001 Canarias, Spain; [email protected] 4 Facultad de Ciencias Básicas, Universidad Católica del Maule, 3466706 Talca, Chile; [email protected] * Correspondence: [email protected] or [email protected] Received: 30 June 2020; Accepted: 19 August 2020; Published: 1 September 2020 Abstract: In this paper, we consider skew-normal distributions for constructing new a distribution which allows us to model proportions and rates with zero/one inflation as an alternative to the inflated beta distributions. The new distribution is a mixture between a Bernoulli distribution for explaining the zero/one excess and a censored skew-normal distribution for the continuous variable. The maximum likelihood method is used for parameter estimation. Observed and expected Fisher information matrices are derived to conduct likelihood-based inference in this new type skew-normal distribution. Given the flexibility of the new distributions, we are able to show, in real data scenarios, the good performance of our proposal. Keywords: beta distribution; centered skew-normal distribution; maximum-likelihood methods; Monte Carlo simulations; proportions; R software; rates; zero/one inflated data 1.
    [Show full text]
  • Lecture 2 — September 24 2.1 Recap 2.2 Exponential Families
    STATS 300A: Theory of Statistics Fall 2015 Lecture 2 | September 24 Lecturer: Lester Mackey Scribe: Stephen Bates and Andy Tsao 2.1 Recap Last time, we set out on a quest to develop optimal inference procedures and, along the way, encountered an important pair of assertions: not all data is relevant, and irrelevant data can only increase risk and hence impair performance. This led us to introduce a notion of lossless data compression (sufficiency): T is sufficient for P with X ∼ Pθ 2 P if X j T (X) is independent of θ. How far can we take this idea? At what point does compression impair performance? These are questions of optimal data reduction. While we will develop general answers to these questions in this lecture and the next, we can often say much more in the context of specific modeling choices. With this in mind, let's consider an especially important class of models known as the exponential family models. 2.2 Exponential Families Definition 1. The model fPθ : θ 2 Ωg forms an s-dimensional exponential family if each Pθ has density of the form: s ! X p(x; θ) = exp ηi(θ)Ti(x) − B(θ) h(x) i=1 • ηi(θ) 2 R are called the natural parameters. • Ti(x) 2 R are its sufficient statistics, which follows from NFFC. • B(θ) is the log-partition function because it is the logarithm of a normalization factor: s ! ! Z X B(θ) = log exp ηi(θ)Ti(x) h(x)dµ(x) 2 R i=1 • h(x) 2 R: base measure.
    [Show full text]
  • Distributions (3) © 2008 Winton 2 VIII
    © 2008 Winton 1 Distributions (3) © onntiW8 020 2 VIII. Lognormal Distribution • Data points t are said to be lognormally distributed if the natural logarithms, ln(t), of these points are normally distributed with mean μ ion and standard deviatσ – If the normal distribution is sampled to get points rsample, then the points ersample constitute sample values from the lognormal distribution • The pdf or the lognormal distribution is given byf - ln(x)()μ 2 - 1 2 f(x)= e ⋅ 2 σ >(x 0) σx 2 π because ∞ - ln(x)()μ 2 - ∞ -() t μ-2 1 1 2 1 2 ∫ e⋅ 2eσ dx = ∫ dt ⋅ 2σ (where= t ln(x)) 0 xσ2 π 0σ2 π is the pdf for the normal distribution © 2008 Winton 3 Mean and Variance for Lognormal • It can be shown that in terms of μ and σ σ2 μ + E(X)= 2 e 2⋅μ σ + 2 σ2 Var(X)= e( ⋅ ) e - 1 • If the mean E and variance V for the lognormal distribution are given, then the corresponding μ and σ2 for the normal distribution are given by – σ2 = log(1+V/E2) – μ = log(E) - σ2/2 • The lognormal distribution has been used in reliability models for time until failure and for stock price distributions – The shape is similar to that of the Gamma distribution and the Weibull distribution for the case α > 2, but the peak is less towards 0 © 2008 Winton 4 Graph of Lognormal pdf f(x) 0.5 0.45 μ=4, σ=1 0.4 0.35 μ=4, σ=2 0.3 μ=4, σ=3 0.25 μ=4, σ=4 0.2 0.15 0.1 0.05 0 x 0510 μ and σ are the lognormal’s mean and std dev, not those of the associated normal © onntiW8 020 5 IX.
    [Show full text]
  • Theoretical Properties of the Weighted Feller-Pareto Distributions." Asian Journal of Mathematics and Applications, 2014: 1-12
    CORE Metadata, citation and similar papers at core.ac.uk Provided by Georgia Southern University: Digital Commons@Georgia Southern Georgia Southern University Digital Commons@Georgia Southern Mathematical Sciences Faculty Publications Mathematical Sciences, Department of 2014 Theoretical Properties of the Weighted Feller- Pareto Distributions Oluseyi Odubote Georgia Southern University Broderick O. Oluyede Georgia Southern University, [email protected] Follow this and additional works at: https://digitalcommons.georgiasouthern.edu/math-sci-facpubs Part of the Mathematics Commons Recommended Citation Odubote, Oluseyi, Broderick O. Oluyede. 2014. "Theoretical Properties of the Weighted Feller-Pareto Distributions." Asian Journal of Mathematics and Applications, 2014: 1-12. source: http://scienceasia.asia/index.php/ama/article/view/173/ https://digitalcommons.georgiasouthern.edu/math-sci-facpubs/307 This article is brought to you for free and open access by the Mathematical Sciences, Department of at Digital Commons@Georgia Southern. It has been accepted for inclusion in Mathematical Sciences Faculty Publications by an authorized administrator of Digital Commons@Georgia Southern. For more information, please contact [email protected]. ASIAN JOURNAL OF MATHEMATICS AND APPLICATIONS Volume 2014, Article ID ama0173, 12 pages ISSN 2307-7743 http://scienceasia.asia THEORETICAL PROPERTIES OF THE WEIGHTED FELLER-PARETO AND RELATED DISTRIBUTIONS OLUSEYI ODUBOTE AND BRODERICK O. OLUYEDE Abstract. In this paper, for the first time, a new six-parameter class of distributions called weighted Feller-Pareto (WFP) and related family of distributions is proposed. This new class of distributions contains several other Pareto-type distributions such as length-biased (LB) Pareto, weighted Pareto (WP I, II, III, and IV), and Pareto (P I, II, III, and IV) distributions as special cases.
    [Show full text]
  • Field Guide to Continuous Probability Distributions
    Field Guide to Continuous Probability Distributions Gavin E. Crooks v 1.0.0 2019 G. E. Crooks – Field Guide to Probability Distributions v 1.0.0 Copyright © 2010-2019 Gavin E. Crooks ISBN: 978-1-7339381-0-5 http://threeplusone.com/fieldguide Berkeley Institute for Theoretical Sciences (BITS) typeset on 2019-04-10 with XeTeX version 0.99999 fonts: Trump Mediaeval (text), Euler (math) 271828182845904 2 G. E. Crooks – Field Guide to Probability Distributions Preface: The search for GUD A common problem is that of describing the probability distribution of a single, continuous variable. A few distributions, such as the normal and exponential, were discovered in the 1800’s or earlier. But about a century ago the great statistician, Karl Pearson, realized that the known probabil- ity distributions were not sufficient to handle all of the phenomena then under investigation, and set out to create new distributions with useful properties. During the 20th century this process continued with abandon and a vast menagerie of distinct mathematical forms were discovered and invented, investigated, analyzed, rediscovered and renamed, all for the purpose of de- scribing the probability of some interesting variable. There are hundreds of named distributions and synonyms in current usage. The apparent diver- sity is unending and disorienting. Fortunately, the situation is less confused than it might at first appear. Most common, continuous, univariate, unimodal distributions can be orga- nized into a small number of distinct families, which are all special cases of a single Grand Unified Distribution. This compendium details these hun- dred or so simple distributions, their properties and their interrelations.
    [Show full text]
  • Bayesian Analysis 1 Introduction
    Bayesian analysis Class Notes Manuel Arellano March 8, 2016 1 Introduction Bayesian methods have traditionally had limited influence in empirical economics, but they have become increasingly important with the popularization of computer-intensive stochastic simulation algorithms in the 1990s. This is particularly so in macroeconomics, where applications of Bayesian inference include vector autoregressions (VARs) and dynamic stochastic general equilibrium (DSGE) models. Bayesian approaches are also attractive in models with many parameters, such as panel data models with individual heterogeneity and flexible nonlinear regression models. Examples include discrete choice models of consumer demand in the fields of industrial organization and marketing. An empirical study uses data to learn about quantities of interest (parameters). A likelihood function or some of its features specify the information in the data about those quantities. Such specification typically involves the use of a priori information in the form of parametric or functional restrictions. In the Bayesian approach to inference, one not only assigns a probability measure to the sample space but also to the parameter space. Specifying a probability distribution over potential parameter values is the conventional way of modelling uncertainty in decision-making, and offers a systematic way of incorporating uncertain prior information into statistical procedures. Outline The following section introduces the Bayesian way of combining a prior distribution with the likelihood of the data to generate point and interval estimates. This is followed by some comments on the specification of prior distributions. Next we turn to discuss asymptotic approximations; the main result is that in regular cases there is a large-sample equivalence between Bayesian probability statements and frequentist confidence statements.
    [Show full text]
  • On Families of Generalized Pareto Distributions: Properties and Applications
    Journal of Data Science 377-396 , DOI: 10.6339/JDS.201804_16(2).0008 On Families of Generalized Pareto Distributions: Properties and Applications D. Hamed1 and F. Famoye2 and C. Lee2 1 Department of Mathematics, Winthrop University, Rock Hill, SC, USA; 2Department of Mathematics, Central Michigan University, Mount Pleasant, Michigan, USA In this paper, we introduce some new families of generalized Pareto distributions using the T-R{Y} framework. These families of distributions are named T-Pareto{Y} families, and they arise from the quantile functions of exponential, log-logistic, logistic, extreme value, Cauchy and Weibull distributions. The shapes of these T-Pareto families can be unimodal or bimodal, skewed to the left or skewed to the right with heavy tail. Some general properties of the T-Pareto{Y} family are investigated and these include the moments, modes, mean deviations from the mean and from the median, and Shannon entropy. Several new generalized Pareto distributions are also discussed. Four real data sets from engineering, biomedical and social science are analyzed to demonstrate the flexibility and usefulness of the T-Pareto{Y} families of distributions. Key words: Shannon entropy; quantile function; moment; T-X family. 1. Introduction The Pareto distribution is named after the well-known Italian-born Swiss sociologist and economist Vilfredo Pareto (1848-1923). Pareto [1] defined Pareto’s Law, which can be stated as N Axa , where N represents the number of persons having income x in a population. Pareto distribution is commonly used in modelling heavy tailed distributions, including but not limited to income, insurance and city size populations.
    [Show full text]
  • Probability Density Function of Non-Reactive Solute Concentration in Heterogeneous Porous Formations ⁎ Alberto Bellin A, , Daniele Tonina B,C
    Journal of Contaminant Hydrology 94 (2007) 109–125 www.elsevier.com/locate/jconhyd Probability density function of non-reactive solute concentration in heterogeneous porous formations ⁎ Alberto Bellin a, , Daniele Tonina b,c a Dipartimento di Ingegneria Civile e Ambientale, Università di Trento, via Mesiano 77, I-38050 Trento, Italy b Earth and Planetary Science Department, University of California, Berkeley, USA c USFS, Rocky Mountain Research Station 322 East Front Street, Suite 401 Boise, ID, 83702, USA Received 20 June 2006; received in revised form 15 May 2007; accepted 17 May 2007 Available online 12 June 2007 Abstract Available models of solute transport in heterogeneous formations lack in providing complete characterization of the predicted concentration. This is a serious drawback especially in risk analysis where confidence intervals and probability of exceeding threshold values are required. Our contribution to fill this gap of knowledge is a probability distribution model for the local concentration of conservative tracers migrating in heterogeneous aquifers. Our model accounts for dilution, mechanical mixing within the sampling volume and spreading due to formation heterogeneity. It is developed by modeling local concentration dynamics with an Ito Stochastic Differential Equation (SDE) that under the hypothesis of statistical stationarity leads to the Beta probability distribution function (pdf) for the solute concentration. This model shows large flexibility in capturing the smoothing effect of the sampling volume and the associated reduction of the probability of exceeding large concentrations. Furthermore, it is fully characterized by the first two moments of the solute concentration, and these are the same pieces of information required for standard geostatistical techniques employing Normal or Log-Normal distributions.
    [Show full text]
  • A Multivariate Beta-Binomial Model Which Allows to Conduct Bayesian Inference Regarding Θ Or Transformations Thereof
    SIMULTANEOUS INFERENCE FOR MULTIPLE PROPORTIONS:AMULTIVARIATE BETA-BINOMIAL MODEL Max Westphal∗ Institute for Statistics University of Bremen Bremen, Germany March 20, 2020 ABSTRACT Statistical inference in high-dimensional settings is challenging when standard unregular- ized methods are employed. In this work, we focus on the case of multiple correlated proportions for which we develop a Bayesian inference framework. For this purpose, we construct an m-dimensional Beta distribution from a 2m-dimensional Dirichlet distribution, building on work by Olkin and Trikalinos(2015). This readily leads to a multivariate Beta- binomial model for which simple update rules from the common Dirichlet-multinomial model can be adopted. From the frequentist perspective, this approach amounts to adding pseudo-observations to the data and allows a joint shrinkage estimation of mean vector and covariance matrix. For higher dimensions (m > 10), the extensive model based on 2m parameters starts to become numerically infeasible. To counter this problem, we utilize a reduced parametrisation which has only 1 + m(m + 1)=2 parameters describing first and second order moments. A copula model can then be used to approximate the (posterior) multivariate Beta distribution. A natural inference goal is the construction of multivariate credible regions. The properties of different credible regions are assessed in a simulation arXiv:1911.00098v4 [stat.ME] 20 Mar 2020 study in the context of investigating the accuracy of multiple binary classifiers. It is shown that the extensive and copula approach lead to a (Bayes) coverage probability very close to the target level. In this regard, they outperform credible regions based on a normal approxi- mation of the posterior distribution, in particular for small sample sizes.
    [Show full text]