On the Beta Prime Prior for Scale Parameters in High-Dimensional Bayesian Regression Models
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Statistical Distributions in General Insurance Stochastic Processes
Statistical distributions in general insurance stochastic processes by Jan Hendrik Harm Steenkamp Submitted in partial fulfilment of the requirements for the degree Magister Scientiae In the Department of Statistics In the Faculty of Natural & Agricultural Sciences University of Pretoria Pretoria January 2014 © University of Pretoria 1 I, Jan Hendrik Harm Steenkamp declare that the dissertation, which I hereby submit for the degree Magister Scientiae in Mathematical Statistics at the University of Pretoria, is my own work and has not previously been submit- ted for a degree at this or any other tertiary institution. SIGNATURE: DATE: 31 January 2014 © University of Pretoria Summary A general insurance risk model consists of in initial reserve, the premiums collected, the return on investment of these premiums, the claims frequency and the claims sizes. Except for the initial reserve, these components are all stochastic. The assumption of the distributions of the claims sizes is an integral part of the model and can greatly influence decisions on reinsurance agreements and ruin probabilities. An array of parametric distributions are available for use in describing the distribution of claims. The study is focussed on parametric distributions that have positive skewness and are defined for positive real values. The main properties and parameterizations are studied for a number of distribu- tions. Maximum likelihood estimation and method-of-moments estimation are considered as techniques for fitting these distributions. Multivariate nu- merical maximum likelihood estimation algorithms are proposed together with discussions on the efficiency of each of the estimation algorithms based on simulation exercises. These discussions are accompanied with programs developed in SAS PROC IML that can be used to simulate from the var- ious parametric distributions and to fit these parametric distributions to observed data. -
Paradoxes and Priors in Bayesian Regression
Paradoxes and Priors in Bayesian Regression Dissertation Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the Graduate School of The Ohio State University By Agniva Som, B. Stat., M. Stat. Graduate Program in Statistics The Ohio State University 2014 Dissertation Committee: Dr. Christopher M. Hans, Advisor Dr. Steven N. MacEachern, Co-advisor Dr. Mario Peruggia c Copyright by Agniva Som 2014 Abstract The linear model has been by far the most popular and most attractive choice of a statistical model over the past century, ubiquitous in both frequentist and Bayesian literature. The basic model has been gradually improved over the years to deal with stronger features in the data like multicollinearity, non-linear or functional data pat- terns, violation of underlying model assumptions etc. One valuable direction pursued in the enrichment of the linear model is the use of Bayesian methods, which blend information from the data likelihood and suitable prior distributions placed on the unknown model parameters to carry out inference. This dissertation studies the modeling implications of many common prior distri- butions in linear regression, including the popular g prior and its recent ameliorations. Formalization of desirable characteristics for model comparison and parameter esti- mation has led to the growth of appropriate mixtures of g priors that conform to the seven standard model selection criteria laid out by Bayarri et al. (2012). The existence of some of these properties (or lack thereof) is demonstrated by examining the behavior of the prior under suitable limits on the likelihood or on the prior itself. -
A Compendium of Conjugate Priors
A Compendium of Conjugate Priors Daniel Fink Environmental Statistics Group Department of Biology Montana State Univeristy Bozeman, MT 59717 May 1997 Abstract This report reviews conjugate priors and priors closed under sampling for a variety of data generating processes where the prior distributions are univariate, bivariate, and multivariate. The effects of transformations on conjugate prior relationships are considered and cases where conjugate prior relationships can be applied under transformations are identified. Univariate and bivariate prior relationships are verified using Monte Carlo methods. Contents 1 Introduction Experimenters are often in the position of having had collected some data from which they desire to make inferences about the process that produced that data. Bayes' theorem provides an appealing approach to solving such inference problems. Bayes theorem, π(θ) L(θ x ; : : : ; x ) g(θ x ; : : : ; x ) = j 1 n (1) j 1 n π(θ) L(θ x ; : : : ; x )dθ j 1 n is commonly interpreted in the following wayR. We want to make some sort of inference on the unknown parameter(s), θ, based on our prior knowledge of θ and the data collected, x1; : : : ; xn . Our prior knowledge is encapsulated by the probability distribution on θ; π(θ). The data that has been collected is combined with our prior through the likelihood function, L(θ x ; : : : ; x ) . The j 1 n normalized product of these two components yields a probability distribution of θ conditional on the data. This distribution, g(θ x ; : : : ; x ) , is known as the posterior distribution of θ. Bayes' j 1 n theorem is easily extended to cases where is θ multivariate, a vector of parameters. -
A Review of Bayesian Optimization
Taking the Human Out of the Loop: A Review of Bayesian Optimization The Harvard community has made this article openly available. Please share how this access benefits you. Your story matters Citation Shahriari, Bobak, Kevin Swersky, Ziyu Wang, Ryan P. Adams, and Nando de Freitas. 2016. “Taking the Human Out of the Loop: A Review of Bayesian Optimization.” Proc. IEEE 104 (1) (January): 148– 175. doi:10.1109/jproc.2015.2494218. Published Version doi:10.1109/JPROC.2015.2494218 Citable link http://nrs.harvard.edu/urn-3:HUL.InstRepos:27769882 Terms of Use This article was downloaded from Harvard University’s DASH repository, and is made available under the terms and conditions applicable to Open Access Policy Articles, as set forth at http:// nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of- use#OAP 1 Taking the Human Out of the Loop: A Review of Bayesian Optimization Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P. Adams and Nando de Freitas Abstract—Big data applications are typically associated with and the analytics company that sits between them. The analyt- systems involving large numbers of users, massive complex ics company must develop procedures to automatically design software systems, and large-scale heterogeneous computing and game variants across millions of users; the objective is to storage architectures. The construction of such systems involves many distributed design choices. The end products (e.g., rec- enhance user experience and maximize the content provider’s ommendation systems, medical analysis tools, real-time game revenue. engines, speech recognizers) thus involves many tunable config- The preceding examples highlight the importance of au- uration parameters. -
Field Guide to Continuous Probability Distributions
Field Guide to Continuous Probability Distributions Gavin E. Crooks v 1.0.0 2019 G. E. Crooks – Field Guide to Probability Distributions v 1.0.0 Copyright © 2010-2019 Gavin E. Crooks ISBN: 978-1-7339381-0-5 http://threeplusone.com/fieldguide Berkeley Institute for Theoretical Sciences (BITS) typeset on 2019-04-10 with XeTeX version 0.99999 fonts: Trump Mediaeval (text), Euler (math) 271828182845904 2 G. E. Crooks – Field Guide to Probability Distributions Preface: The search for GUD A common problem is that of describing the probability distribution of a single, continuous variable. A few distributions, such as the normal and exponential, were discovered in the 1800’s or earlier. But about a century ago the great statistician, Karl Pearson, realized that the known probabil- ity distributions were not sufficient to handle all of the phenomena then under investigation, and set out to create new distributions with useful properties. During the 20th century this process continued with abandon and a vast menagerie of distinct mathematical forms were discovered and invented, investigated, analyzed, rediscovered and renamed, all for the purpose of de- scribing the probability of some interesting variable. There are hundreds of named distributions and synonyms in current usage. The apparent diver- sity is unending and disorienting. Fortunately, the situation is less confused than it might at first appear. Most common, continuous, univariate, unimodal distributions can be orga- nized into a small number of distinct families, which are all special cases of a single Grand Unified Distribution. This compendium details these hun- dred or so simple distributions, their properties and their interrelations. -
9 Introduction to Hierarchical Models
9 Introduction to Hierarchical Models One of the important features of a Bayesian approach is the relative ease with which hierarchical models can be constructed and estimated using Gibbs sampling. In fact, one of the key reasons for the recent growth in the use of Bayesian methods in the social sciences is that the use of hierarchical models has also increased dramatically in the last two decades. Hierarchical models serve two purposes. One purpose is methodological; the other is substantive. Methodologically, when units of analysis are drawn from clusters within a population (communities, neighborhoods, city blocks, etc.), they can no longer be considered independent. Individuals who come from the same cluster will be more similar to each other than they will be to individuals from other clusters. Therefore, unobserved variables may in- duce statistical dependence between observations within clusters that may be uncaptured by covariates within the model, violating a key assumption of maximum likelihood estimation as it is typically conducted when indepen- dence of errors is assumed. Recall that a likelihood function, when observations are independent, is simply the product of the density functions for each ob- servation taken over all the observations. However, when independence does not hold, we cannot construct the likelihood as simply. Thus, one reason for constructing hierarchical models is to compensate for the biases—largely in the standard errors—that are introduced when the independence assumption is violated. See Ezell, Land, and Cohen (2003) for a thorough review of the approaches that have been used to correct standard errors in hazard model- ing applications with repeated events, one class of models in which repeated measurement yields hierarchical clustering. -
A Class of Generalized Beta Distributions, Pareto Power Series and Weibull Power Series
A CLASS OF GENERALIZED BETA DISTRIBUTIONS, PARETO POWER SERIES AND WEIBULL POWER SERIES ALICE LEMOS DE MORAIS Primary advisor: Prof. Audrey Helen M. A. Cysneiros Secondary advisor: Prof. Gauss Moutinho Cordeiro Concentration area: Probability Disserta¸c˜ao submetida como requerimento parcial para obten¸c˜aodo grau de Mestre em Estat´ısticapela Universidade Federal de Pernambuco. Recife, fevereiro de 2009 Morais, Alice Lemos de A class of generalized beta distributions, Pareto power series and Weibull power series / Alice Lemos de Morais - Recife : O Autor, 2009. 103 folhas : il., fig., tab. Dissertação (mestrado) – Universidade Federal de Pernambuco. CCEN. Estatística, 2009. Inclui bibliografia e apêndice. 1. Probabilidade. I. Título. 519.2 CDD (22.ed.) MEI2009-034 Agradecimentos A` minha m˜ae,Marcia, e ao meu pai, Marcos, por serem meus melhores amigos. Agrade¸co pelo apoio `aminha vinda para Recife e pelo apoio financeiro. Agrade¸coaos meus irm˜aos, Daniel, Gabriel e Isadora, bem como a meus primos, Danilo e Vanessa, por divertirem minhas f´erias. A` minha tia Gilma pelo carinho e por pedir `asua amiga, Mirlana, que me acolhesse nos meus primeiros dias no Recife. A` Mirlana por amenizar minha mudan¸cade ambiente fazendo-me sentir em casa durante meus primeiros dias nesta cidade. Agrade¸comuito a ela e a toda sua fam´ılia. A` toda minha fam´ılia e amigos pela despedida de Belo Horizonte e `aS˜aozinha por preparar meu prato predileto, frango ao molho pardo com angu. A` vov´oLuzia e ao vovˆoTunico por toda amizade, carinho e preocupa¸c˜ao, por todo apoio nos meus dias mais dif´ıceis em Recife e pelo apoio financeiro nos momentos que mais precisei. -
On the Robustness to Misspecification of Α-Posteriors and Their
Robustness of α-posteriors and their variational approximations On the Robustness to Misspecification of α-Posteriors and Their Variational Approximations Marco Avella Medina [email protected] Department of Statistics Columbia University New York, NY 10027, USA Jos´eLuis Montiel Olea [email protected] Department of Economics Columbia University New York, NY 10027, USA Cynthia Rush [email protected] Department of Statistics Columbia University New York, NY 10027, USA Amilcar Velez [email protected] Department of Economics Northwestern University Evanston, IL 60208, USA Abstract α-posteriors and their variational approximations distort standard posterior inference by downweighting the likelihood and introducing variational approximation errors. We show that such distortions, if tuned appropriately, reduce the Kullback-Leibler (KL) divergence from the true, but perhaps infeasible, posterior distribution when there is potential para- arXiv:2104.08324v1 [stat.ML] 16 Apr 2021 metric model misspecification. To make this point, we derive a Bernstein-von Mises theorem showing convergence in total variation distance of α-posteriors and their variational approx- imations to limiting Gaussian distributions. We use these distributions to evaluate the KL divergence between true and reported posteriors. We show this divergence is minimized by choosing α strictly smaller than one, assuming there is a vanishingly small probability of model misspecification. The optimized value becomes smaller as the the misspecification becomes -
Concentration Inequalities from Likelihood Ratio Method
Concentration Inequalities from Likelihood Ratio Method ∗ Xinjia Chen September 2014 Abstract We explore the applications of our previously established likelihood-ratio method for deriving con- centration inequalities for a wide variety of univariate and multivariate distributions. New concentration inequalities for various distributions are developed without the idea of minimizing moment generating functions. Contents 1 Introduction 3 2 Likelihood Ratio Method 4 2.1 GeneralPrinciple.................................... ...... 4 2.2 Construction of Parameterized Distributions . ............. 5 2.2.1 WeightFunction .................................... .. 5 2.2.2 ParameterRestriction .............................. ..... 6 3 Concentration Inequalities for Univariate Distributions 7 3.1 BetaDistribution.................................... ...... 7 3.2 Beta Negative Binomial Distribution . ........ 7 3.3 Beta-Prime Distribution . ....... 8 arXiv:1409.6276v1 [math.ST] 1 Sep 2014 3.4 BorelDistribution ................................... ...... 8 3.5 ConsulDistribution .................................. ...... 8 3.6 GeetaDistribution ................................... ...... 9 3.7 GumbelDistribution.................................. ...... 9 3.8 InverseGammaDistribution. ........ 9 3.9 Inverse Gaussian Distribution . ......... 10 3.10 Lagrangian Logarithmic Distribution . .......... 10 3.11 Lagrangian Negative Binomial Distribution . .......... 10 3.12 Laplace Distribution . ....... 11 ∗The author is afflicted with the Department of Electrical -
Package 'Extradistr'
Package ‘extraDistr’ September 7, 2020 Type Package Title Additional Univariate and Multivariate Distributions Version 1.9.1 Date 2020-08-20 Author Tymoteusz Wolodzko Maintainer Tymoteusz Wolodzko <[email protected]> Description Density, distribution function, quantile function and random generation for a number of univariate and multivariate distributions. This package implements the following distributions: Bernoulli, beta-binomial, beta-negative binomial, beta prime, Bhattacharjee, Birnbaum-Saunders, bivariate normal, bivariate Poisson, categorical, Dirichlet, Dirichlet-multinomial, discrete gamma, discrete Laplace, discrete normal, discrete uniform, discrete Weibull, Frechet, gamma-Poisson, generalized extreme value, Gompertz, generalized Pareto, Gumbel, half-Cauchy, half-normal, half-t, Huber density, inverse chi-squared, inverse-gamma, Kumaraswamy, Laplace, location-scale t, logarithmic, Lomax, multivariate hypergeometric, multinomial, negative hypergeometric, non-standard beta, normal mixture, Poisson mixture, Pareto, power, reparametrized beta, Rayleigh, shifted Gompertz, Skellam, slash, triangular, truncated binomial, truncated normal, truncated Poisson, Tukey lambda, Wald, zero-inflated binomial, zero-inflated negative binomial, zero-inflated Poisson. License GPL-2 URL https://github.com/twolodzko/extraDistr BugReports https://github.com/twolodzko/extraDistr/issues Encoding UTF-8 LazyData TRUE Depends R (>= 3.1.0) LinkingTo Rcpp 1 2 R topics documented: Imports Rcpp Suggests testthat, LaplacesDemon, VGAM, evd, hoa, -
Parametric Representation of the Top of Income Distributions: Options, Historical Evidence and Model Selection
PARAMETRIC REPRESENTATION OF THE TOP OF INCOME DISTRIBUTIONS: OPTIONS, HISTORICAL EVIDENCE AND MODEL SELECTION Vladimir Hlasny Working Paper 90 March, 2020 The CEQ Working Paper Series The CEQ Institute at Tulane University works to reduce inequality and poverty through rigorous tax and benefit incidence analysis and active engagement with the policy community. The studies published in the CEQ Working Paper series are pre-publication versions of peer-reviewed or scholarly articles, book chapters, and reports produced by the Institute. The papers mainly include empirical studies based on the CEQ methodology and theoretical analysis of the impact of fiscal policy on poverty and inequality. The content of the papers published in this series is entirely the responsibility of the author or authors. Although all the results of empirical studies are reviewed according to the protocol of quality control established by the CEQ Institute, the papers are not subject to a formal arbitration process. Moreover, national and international agencies often update their data series, the information included here may be subject to change. For updates, the reader is referred to the CEQ Standard Indicators available online in the CEQ Institute’s website www.commitmentoequity.org/datacenter. The CEQ Working Paper series is possible thanks to the generous support of the Bill & Melinda Gates Foundation. For more information, visit www.commitmentoequity.org. The CEQ logo is a stylized graphical representation of a Lorenz curve for a fairly unequal distribution of income (the bottom part of the C, below the diagonal) and a concentration curve for a very progressive transfer (the top part of the C). -
Prior Distributions for Variance Parameters in Hierarchical Models
Bayesian Analysis (2006) 1, Number 3, pp. 515–533 Prior distributions for variance parameters in hierarchical models Andrew Gelman Department of Statistics and Department of Political Science Columbia University Abstract. Various noninformative prior distributions have been suggested for scale parameters in hierarchical models. We construct a new folded-noncentral-t family of conditionally conjugate priors for hierarchical standard deviation pa- rameters, and then consider noninformative and weakly informative priors in this family. We use an example to illustrate serious problems with the inverse-gamma family of “noninformative” prior distributions. We suggest instead to use a uni- form prior on the hierarchical standard deviation, using the half-t family when the number of groups is small and in other settings where a weakly informative prior is desired. We also illustrate the use of the half-t family for hierarchical modeling of multiple variance parameters such as arise in the analysis of variance. Keywords: Bayesian inference, conditional conjugacy, folded-noncentral-t distri- bution, half-t distribution, hierarchical model, multilevel model, noninformative prior distribution, weakly informative prior distribution 1 Introduction Fully-Bayesian analyses of hierarchical linear models have been considered for at least forty years (Hill, 1965, Tiao and Tan, 1965, and Stone and Springer, 1965) and have remained a topic of theoretical and applied interest (see, e.g., Portnoy, 1971, Box and Tiao, 1973, Gelman et al., 2003, Carlin and Louis, 1996, and Meng and van Dyk, 2001). Browne and Draper (2005) review much of the extensive literature in the course of comparing Bayesian and non-Bayesian inference for hierarchical models. As part of their article, Browne and Draper consider some different prior distributions for variance parameters; here, we explore the principles of hierarchical prior distributions in the context of a specific class of models.